Sysadmin Field Notes - Software Development Archives

October 4, 2009

When trying to setup the sql persistence database on SQL Express 2005 (the one that comes with visual studio), I got one of two errors. If I clicked on the drop down to select a database I got "Failed to query a list of database names from the SQL Server." blah blah blah "Named Pipes Provider, error: 40 - Could not open a connection to SQL Server."

If I just went ahead without selecting a database, I got a large stack trace with basically the same error:

ystem.Web.HttpException: Unable to connect to SQL Server database. ---> System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)...

The advice on the internet for this error walks you through editing the surface area configuration to enable remote connections via TCP and named pipes. I had already done that. There was some mention of SQL Express being a named instance and requiring a different connection string, but I'm not familiar enough with any of these tools to know off hand how to make aspnet_regsql honor any of those.

Finally figured it out, on that first dialog box where it has your hostname for server, just add a backslash and SQLEXPRESS. This will get it to the right named instance. So for example, "MYHOSTNAME" becomes "MYHOSTNAME\SQLEXPRESS". As soon as I did that, the errors went away.

Update:

The wizard mode actually apparently won't setup sql state, just some other services. So you have to do command line. This is how to pass what you want at the command line:

aspnet_regsql -S .\sqlexpress -E -ssadd -sstype p

Was the magic incantation that worked for me to create the ASPState database on my SQL Express database.

June 15, 2009

I always forget this, so here it is so I'll remember. For some reason search engines don't seem to return good results here.

If you've got an ugly, un-tabbed, unformatted blob of xml, just paste it into emacs and do:

M-x xml-mode
M-x sgml-pretty-print

And will be nicely indented. Presumably this requires a modernish Emacs.

February 12, 2009

The perception of software developers is that when asked to produce something for a client, they respond with unintelligible questions. "Do you want me to do it in Ruby on Rails?" "Should I use MySql or PostgreSQL?" "Unix or Windows?"

That's given us a bit of a bad rap (as it should). We're not the only industry that does it though. When I go in for a haircut, they very rarely ask me what I want hair to look like, they focus on the tools and techniques as well. Questions like scissors or clippers, what # guide for clippers, should I use the thinning shears, how many inches to take off, etc....none of those I care at all about or ever remember.

I really just want to express my problem in terms I can understand, and then let the profession translate that into how best to deliver a solution. So next time a customer asks you for something, make sure you're helping them describe what they want, and not how to get there.

November 9, 2008

Something just whizzed by my Google reader that was a link to a longer article saying that Agile was more about the principles than the methodology.

I think that's exactly true. It's about setting up a team that's always looking for things that could derail the project, bringing those issues up in a collaborative environment, and (this is the most important thing), explaining them to the customer in a way they understand.

It's not enough to find a problem, you have to make that problem front and center, whether that problem is that you're building the wrong thing, or that what you're building isn't going to be ready. Whereas traditional software development is caveat emptor, agile turns that around.

If your team can keep that in mind, you're going to be trusted by your customer regardless of methodologies. But by funneling the majority of contact with your customer through large documents, you're making it more difficult for both sides to find and communicate the misunderstandings.

August 5, 2008

There's not a great google entry already for this, so I thought I'd capture it. If you're a dumbass like me and keep forgetting that when you are creating accessors, they are symbols at that point and not variables, this is the mysterious error you get:

filename.rb:1: nil is not a symbol (TypeError)

     from filename.rb:1

This happens when you do this in an object:

attr_accessor @attribute_name

Instead of the correct:

attr_accessor :attribute_name

I'm putting here more as a note to myself more than anything (maybe I'll just stop forgetting...naaahhh).

May 17, 2008

If only things were actually this funny. It's actually not at all funny. Luckily, the only version of linux I have hanging around is ancient and not one of the ones affected.

November 11, 2007

Apparently, even the International Institute of Business Analysis doesn't know what a business analyst is.

In case it changes, here's a screenshot of the page entitled What is a business analyst?:

I'm sure it's just a mistake or unfinished page, but you have to admit it's pretty humorous.

November 4, 2007

Oren blogs today about Castle Style Errors.

I admit, those are good error messages; someone put some thought into them and that's the most important thing.

These are great messages for developers, but are they as appropriate for the code once it's running in production and being maintained by the operations team? A team that might not be at all familiar with the code or the libraries in use.... Now, I don't know what causes these errors because I haven't encountered them, so these might be fine.

But a lot of times when the exceptions are very developer specific, they can be misleading. It requires a 2nd layer of thought; Why would this be broken in prod? It's okay if the developer-specific message tags along, but it's important to have any root cause (missing database table,broken db, connection error, etc...) displayed as obviously as possible.

This is why error messages for server-side systems are a pain. It's very hard to get right, since there are multiple audiences. And knowing for certain what the root cause is when you're writing the code tends to be difficult.

Update: Typos fixed and minor reword of a sentence for clarity.

July 31, 2007

Here's a very nice explanation of the Law of Demeter.

In thinking about this and its relation with Domain Driven Design makes me think of something that initially seems like a good guideline. You should watch very carefully for "Demeter" violations between aggregates, but within an aggregate they are probably less dangerous.

Focusing on custom messages between your aggregates in DDD would seem to be a big help in two ways. First by helping root out your aggregate boundaries during design, by looking for high level behavior conversations that might exist. Second by helping to break data dependencies between aggregates that might otherwise be very tempting to introduce when trying to persist that group of objects.

I think paying more attention to obeying the Law of Demeter between aggregates and worrying about it less within the group might be a good way of keeping balance between creating custom messages for everything and re-using data attributes inappropriate in a way that makes code fragile.

June 20, 2007

It appears that I disagree about architecture with a fair bit of my industry. I'm enjoying the though exercise of trying to find a middle ground on my views. It's this post, Mort and the Economics of Unmaintainable Code, that highlights this most effectively.

If I've got it right the idea expressed here is that smart enough people can create frameworks that limit the damage that the "Mort's" of the world can do.

I completely disagree with that, because I've never seen it work. The code that Mort used to write when all we had was plain old CGI and you manually had to parse all your form fields out of the request was just as bad as the code Mort writes today with the web client software factory, or Rails, or any other tool that's out there.

The problem is that programming is not a constrained space. You can hand someone a perfectly simple and reasonable abstraction, and they can completely misunderstand and obliterate it. And it's not free to fix. Because all that work Mort did with the client to figure out what they were building and get it into the code gets intermixed with Mort's terrible abstractions.

Mort will not constrain him or herself to directly use the framework. Let's say Mort is consuming a magical service that works perfectly for his application. (We'll leave aside the question of how an organization that hires Morts so freely to build applications manages to avoid hiring them to build the service). All that code that interacts with the services, while it may be just a tiny, constrained chunk in the grand architectual scheme, will expand to dominate the system.

Because remember human nature. Mort doesn't know he/she is a Mort. The less skilled don't realize they are less skilled. [pdf]. Yes, that very much includes me. :-) Mort will ignore your carefully crafted architecture because he/she thinks they can do better. Or just don't care.

Scaling software is a fabulously hard problem. Unsolveable in the whole I believe, so every solution has to be tailored for the people, the situation, and the organization. Until there's a major technological breakthrough (like artificial intelligence, assuming it's possible), there is not going to be a silver bullet.

Which is why in the end I don't find much value in tools that try to solve it for everyone. The whole simplification of "Mort" doesn't even hold. There are a million sub-varieties of Mort. A talented lead/architect can judge them all individually, and figure out how to put them to best use.

Again, this doesn't mean you just cast aside Mort, they build a lot of (sort of) working software the business depend on, in ways that aren't even obvious. So it's good that people build tools targeted towards them. While ASP.net 2.0 code for a web app from Mort is just as bad as hand-rolled CGI's in Perl from a Mort, there's a lot more value in it.

But code is not now, and has never been "free". Quite the opposite in my experience; the most business value is wrapped up in the worst code. Extracting it takes tons of time and careful effort. I've seen maybe 20-30 examples of reasonably sized systems that tried to constrain the damage that Morts could do, and all failed miserably. I'll need to see a pretty wild streak of successes before my opinion changes.

June 20, 2007

Martin Fowler today has tossed out a hypothesis on the value of design.

Here's my anecodtal reference point. A large app we had on our platform was constructed very quickly, with shortcuts, to meet a fast approaching deadline that would not be extended. It started with a reasonable design and some good ideas, but time constraints forced the developers away from their design.

Almost exactly as that app launched to the public, work started on the next generation of that application. This time, still under time constraints, but with a larger more experieinced team that was able to make much fewer sacrifices. The codebases quickly diverged, and within one year the first one was considered to be extremely confusing. It was difficult or impossible to even just fix bugs or troubleshoot production problems. The remaining functionality that the first had was moved to the second.

The second is still around today, the team has turned over or been reassigned several times, yet it's still possible to add meaningful features in a quarter release cycle, and the app performs very well on production.

For me, attention to detail, design, domain driven design, and focusing on readable code are clearly worth it.

June 16, 2007

So there's weird dust-up in the .NET community regarding some fictional persona named Mort. Like Oren Eini, I can't seem to figure out what in the world the argument is. Mort is good? Mort is bad? Mort is "Agile"?

The descriptions of Mort I've seen don't indicate to me that "Mort" has ever heard of, let alone used TDD, domain specific languages, continuous integration, etc. He/she just isn't interested. There's a million things Mort could do today to improve his/her code, but doesn't, either because he/she is not inclined or able to.

Here's what I think the real issue is. Tools can't fix that. Assuming I'm understanding the persona properly (which is highly likely that I'm not, since Sam and Nick work for the same company and don't agree on the definition), Mort will write bad code no matter what tools he/she has. The only thing that can help Mort is targeted instruction by a talented teacher.

If you add in a unit tester to the default package, Mort will write bad, hard to maintain unit tests that are more trouble than they are worth. If you add continuous integration, Mort will only check-in once every two weeks because he/she doesn't want to break the build. Mort needs peer pressure and mentorship to get better, not more tools in the toolbox.

Update: On a re-read, this sounds elitist, but that's not what I was getting at. Mort's do a lot work for corporate America and elevating their level (and identifying/driving out the really terrible ones) is important, maybe even critical to our industry improving. I just think the best way to do that is by attracting talented mentors and leads to the platform that can then bring up their teams, rather than trying to suddenly and spontaneously get Morts to change their behavior for the better just because a new tool popped up in Visual Studio.

June 10, 2007

Funny. This week I got a bunch of hits to my pictures from last year's mather airshow whereas last week I got none. Okay, easy enough, this weekend is this year's airshow, makes sense. Weird part was that -all- of the hits were from Yahoo...not Google. Usually yahoo barely registers, this week it was 10% of the incoming.

Searching for "mather airshow" on yahoo, for some reason, turns my page up #1, whereas on Google I can't even find it.

Should that page really be #1 on yahoo for that search? Ah, I see, "air show" is two words, so I've typojacked it. Google's a lot closer in it's results for airshow.

Interestingly, yahoo suggests: "california capital airshow, mather airport, mather field " which is spot on. The first is the real name of the air show, and takes you right to the front page. Google says "did you mean" and separates the words, but still doesn't have a link to the show's actual site in the top 10.

So I guess the moral is that the mather air show people need to apply a little SEO.

Can we imply that air show folk use Yahoo more, since even though I'm #1, that's still a way, way disproportionate number of yahoo users. I'm #2 for "rob meyer" on yahoo and #2 on google, yet google wins normally by a 10:1 ratio. Seems like a safe call that yahoo is way out of proportion here. Weird.

May 31, 2007

Jamie Cansdale's problems with Microsoft are why, as much I've come to enjoy working with some of the Microsoft technologies over the past two years, I suspect the developer relationship is always going to be dysfunctional. Which, put simply, makes developing on the platform much less fun. It's exactly what Martin Fowler observed about Microsoft's Ruby efforts.

Less fun platforms are harder to attract talent to. It may not matter for most IT shops, which are their bread and butter. But it certainly makes it un-fun enough for me to want to avoid it unless it's a mandate.

So maybe in the short term it doesn't matter that a ton of Thoughtwork's business is moving to Ruby. The immediate influence isn't going to be huge. But guess what? Those "alpha geeks" (and oooh...how I dislike that term) is your next generation of Technology VP's, directors of engineering, and CTO's.

It will take time, but without improving the way it treats the open source development community, I think Microsoft is eventually going to find itself wondering why the CTO's and decision makers started asking better questions, and why they can't seem to close deals on that golf course anymore.

Update: I should mention that I'm not 100% sure Jamie can (or should) win this fight. My point is simply that it's only in an ecosystem dominated by a single company that sets all direction, without much influence from the community, can this sort of thing happen at all. The problem isn't that the TestDriven.NET add on works or doesn't work with Visual Studio Express, the problem is that there is no other IDE. Here I think Jamie's efforts to bring more functionality to the entry level offerings is noble, if not necessarily completely legal or authorized.

March 18, 2007

Martin Folwer just referenced an eBay architecture decision to avoid transactions and the consequences that arise.

Being transactionless doesn't freak me out too much, because really most of the work that my programs are doing involve crossing a system boundary. A message to the mainframe here, a couple of web service calls, a write to a database. Pretty much all we can do is think long and hard about sequencing, and what do when any individual step fails. Which for our application (luckily) is generally "ship a message off to someone who can manually fix the parts that didn't go."

But we certainly don't have anything like distributed transactions, and I'm not sure I even believe in them.

I

December 8, 2006

Leopard Technology Series for Developers

"Leopard embraces
two other highly dynamic languages for use in building Cocoa applications: Ruby and Python. These
two languages are an excellent fit for integrating with the Cocoa frameworks, and they both have
high quality bridges to Objective-C. These bridges allow you to mix and match Objective-C, Ruby, and
Python, allowing you to choose the best tool for the job at hand"

Very cool, officially supported scripting Cocoa bindings...it sounds like Cocoa might be a fun place to develop. I had learned enough to be dangerous before, but as all technology things have fallen off my free-time radar, I sorta lost the thread. This might inspire me to pick it up again and dabble in both Cocoa and Ruby.

The rest of the article is pretty cool tool; it's kind of interesting to watch the evolution of such high level but still systemwide APIs.

October 25, 2006

Dave Churchville has a pretty good response to Steve Yegge's rant on good agile vs. bad agile.

This line: "He warns of Agile proponents "being slippery" by taking credit for any good effects of an implementation, but explaining away bad effects as "you didn't do it right"." made me think.

Of all the problems "waterfall" (which is a bit of a straw man anyway I think, but let's use it for now anyway) methods has caused, people just tell you it's because you're not doing it right.

You didn't get all the requirements.

You didn't anticipate this change and plan for it with a flexible design.

You missed this on your test plan.

In short, "you're not doing it right."

Let's face it, if you're lucky enough to have a critical mass of good developers, testers, customers, and engineers, you can succeed with almost any process.

October 10, 2006

Charles Miller posts a missive regarding the value of commenting performance hacks.

Included is a perfect example of a reasonable comment. It leads me to a very simple guideline:

Comments should contain meta-information about the code in question.

Things that make sense to comment are your state of mind when writing something, other approaches you tried, location it was borrowed from, protocol reference material, etc...Stuff that helps describe the code itself, and why it is what it is.

Most other sorts of comments should be looked at as warning signs. I'm not saying that all other comments should be removed; but if you have comments that indicate calling order dependencies, you owe it to yourself to try and figure out a way to express the API in a way that makes those dependencies implicit (or removes them altogether).

August 24, 2006

Joel on Software is annoyed that everything that comes out of Harvard Business School is "wishy-washy".

I'm surprised at that. The world is a wishy-washy place. A developer knows that. Rules were made to be broken, every situation is different. One thing that generally works in some situations then won't work in others. If the same thing worked every time, then both development and management would be trivial tasks.

August 9, 2006

From Larry Osterman again, I just can't help but comment on his link from today, Dare Obasanjo aka Carnage4Life - Amazon Developer on Replacing Operations with Developers.

I think this is a bit of a simplistic view. The idea that you had your notes to the operations team in the morning and they find the root cause is okay, if and only if your operations team has the time and is capable of that. In many places, as soon as they see a stack trace in the logs, that automatically means it's for a developer to look at and they throw it right back. Or, they have their own new "most important thing", and they don't look at it either.

For operations to be able to follow up on issues, they have to have a few developer-types on their team.

For development to not build sites and services that suck total ass to maintain, they have to have a few operations-minded people on their team.

I don't think you can replace operations with developers, but you have to have a system where the developers feel the pain. When schedules get cramped, it's all too easy to let stuff out that will cause production problems. Unless that pain is reflected onto the individuals responsible, at least a little bit, you're going to have disfunction.

There should be division of labor, but it should be much blurrier than it usually is.

I find a good method is to have the developers get paged (or at least emailed at first) every single error message out of the logs (scrubbed if necessary). Then they know what's going on in prod, and it causes them enough grief that they are motivated to fix it (although the best are motivated by personal pride also).

The one piece often left out in this equation is to have your developers have enough (read only please) access to prod, or prod log data that they can actually investigate the problems that come down the pipe.

On the flip side, at least a few people in ops should absolutely understand the code enough to dig in a bit, assign problems to the right developers, and fix the occasional problems. If it's just a big black box, it's almost impossible to troubleshoot properly.

No pain, no gain. :-)

August 9, 2006

I've lightly covered regions before but since Google has no appropriate entry for "Regions considered harmful" Neil suggested that I write one.

First the background, what's a region? As specified in the grammar, regions are pretty innocuous. They have no semantic meaning, the simply identify a region of code. Seems easy enough right?


#region some hunk of code
foo = this.bar();
bar = new FooBar();
#endregion


The default behavior of Visual Studio is to collapse these regions so the code under them is not visible. People use them to surround public methods, private variables, or regions of interesting code, etc... It's often used to make long classes "more readable" because now the whole class fits on a single screen by default. Why are the bad? One simple reason.

Regions encourage longer and more complex methods and classes

The class or method doesn't really fit on one page, it's just collapsed. It still might be 5000 lines. What's more, is that you have to hunt around in the regions to find out where the real complexity might be. Or just expand them all. Then you realize you're dealing with a 5,000 line class that desperately needs refactoring.

This goes double for methods. I've seen things like this:


if (thing.ShouldHaveFoo()) {
thing.DoThingWith(x);
[do more]
}


(the brackets are my attempt to duplicate the little box and + sign visual studio puts around regions when they are collapsed)

That looks very simple and easy right? But what happens if you expand the Do More region and find that it's actually 5 levels of nested loops and case statements? That code is still there, it's just hidden. I've seen even more pathological examples where the regions span portions of nestings, or there are several regions to hide the innermost bits of a loop.

None of this removes complexity. It just hides its. Like a rug you can sweep dirt under. That's not a great analogy though, because I don't really care if there's dirt under my rug, but I do care if there's hidden complexity in my programs.

Bottom line is that if you can semantically define a set of operations with a logical name that makes sense to people, make it a method. Now it can be re-used, examined, debugged more obviously. If you say "calling a method here would be too expensive," then you better have a comment above the region indicating so.

What you're really doing when you mix regions into complex code is probably mixing abstraction layers, which is bad. Since there's no scoping, it's very easy to use variables outside of the region, which is going to be very confusing to anyone trying to read the code with outlining enabled.

Since outlining (collapsing regions) is enabled by default, that means most programmers will miss this hidden complexity. It's very easy to miss a collapsed region. I've talked to people who have been unable to find a bug, because they didn't notice that half the code was hidden in one line in an unexpected region. So if everyone just has to disable outlining by default or turn it off to read your class, why add regions in the first place?

Let me handle some of the most likely justifications for regions.

Aren't regions useful for hiding generated code?

I would generally say this is probably okay, but wouldn't you rather have the generated code completely separate? I'd rather have whole files marked "off limits" to prying eyes, since life is easier if you can just regenerate whole files.

ASP.net 1.1 uses regions around the designer-introduced code for forms, but they have since moved to partial compilation, which as I understand it keeps the generated code out of your files. Much preferred. So with generated code, regions are just hiding things in a less than ideal way.

Regions are handy for marking the public API of a class

Here I'd probably agree with you, although I've never seen anyone do this before. And it's not terribly useful. If I just want to see the public API, then Visual Studio or Resharper is going to show me that. I'm probably looking at the class because I want to see the implementation, and that just means I'll be expanding a whole bunch of regions.

Typically people have regions like "public methods", "public properties", "private methods", "internal methods", etc...Who cares? If I've resorted to looking at the implementation of a class in my project, it's because I need to see how it's implemented. I certainly am not interested in the distinction between public methods and public properties.

So let's recap:

1. You can't automatically refactor a region,
2. Regions don't abstract complexity away, they just hide it,
3. they encourage longer classes/ethods since it can "look" like the class or method is shorter.

And on the pro-side, they might occasionally be useful for marking the public api of a class, but that's not really necessary all that often. And besides, everyone's going to be turning off regions altogether (hopefully), so they won't see the hiding anyway.

So anytime you go to write a #region, think of it as a poor man's method or subroutine, do yourself a favor and stop. Actually abstract out that functionality into a method with a clear interface. Maintenance programmers everywhere will thank you.

Bonus tip to disable outlining in VS 2003 forever:

First, make sure you only have one copy of Visual studio running (last one you close wins the preferences battle). Go to Tools->Options. Select Text Editor->C#. Unselect "Enter outlining mode when files open". Now exit visual studio to make sure it saves, and you never have to expand a region again.

August 7, 2006

Larry's definitely at least partially right; this sort of thing is just the sort of ethical hairball that keeps software development from growing into a real profession. At the same time, activities like this have definite value...

This is my $.02 on Larry Osterman's latest comments on the Apple contractors who had their small app canceled, but kept coming back unpaid to finish it anyway.

Certainly back in 1994 new features in Windows didn't require a threat model analysis at Microsoft...:-)

At any rate, I'm torn on this one. For one, I don't remember if they said it was going to be included in the base install or just on the disc. Just included on the disk in the unsupported section in my mind isn't that big a deal. I've found some pretty weird stuff in those directories over time on original install media from all kinds of companies. Installed on every system as part of the base install, that could be a bit different.

Also, legal or not, good idea or not, this was a pretty cool hack, and a good deal of fun. Things like this give companies (and products) identities, and that in many ways is preferable to soulless processes churning out lifeless bits.

No engineer wants to work on or use a system with no controls that people just slap all sorts of random stuff into. At the same time, it's no fun to work in a tightly-controlled bureaucracy. This is the trouble I think Microsoft finds itself in right now. By necessity because of the market share and sensitive systems that run windows, something like this could never be allowed to happen. But that sterility doesn't lend itself to passionate engineers, or developers. (just my $.02).

I don't think anyone should emulate the things they did, or that they are advocating anyone emulating this behavior. But it's not obvious to me how a team keeps it's culture, folklore, and personality when there's enough process to completely eliminate all possibility of anything like this ever happening.

June 26, 2006

An article on ZDNet asks » When does SOA stop being worthwhile? | Service-Oriented Architecture | ZDNet.com

I'm just tired and cynical right now so I would have asked the opposite question, when does SOA start being valuable?

May 31, 2006

Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems. - Jamie Zawinski

I love regular expressions. I like their pseudo computational theory roots (regular expressions as we think of them are not exactly "regular" expressions in the mathematical sense, but its mostly the same ideas), and they are always fun to write. I think that grep is the single greatest peice of software ever written.

That being said, I generally don't like to get too fancy with regexps in production code. For one, as we discovered, they can end up with exponential running time. That's bad when putting in an invalid email address in a form field acts as a denial of service attack on your site. They make great filters, and pretty good matchers when you need to pull out part of a string. And if you're writing a one-line shell script for log analysis, they are wonderful.

But in production code, regular expressions tend to violate the most important general principle for enterprise development, readability. Within a single, simple looking regex, can lie a ton of defects. When someone goes to fix a pattern that a regex is letting through, it's very difficult to ensure you didn't break other patterns.

Of course, it's not necessary to hand-write a full blown parser for most things. Simple regexes are good, so sometimes some combination of basic hand parsing and regular expressions can be very effective. There's no shame in using a few simple expressions instead of a big complicated one. They also don't do the best job of solving every problem.

For example, say you're trying to get the first 400 characters of long string, but you want it bounded at a word boundary, running slightly over 400 characters if necessary. The regex for that is something lke: /^(.{400}[^\s]*)\s/s (perl format). Now which is more readable, that thing, or:

firstWordBoundaryAfter400Characters = fooString.IndexOf(" ", 400);
return fooString.SubString(0, firstfirstWordBoundaryAfter400Characters + 1);

I know which one I'd rather fix a defect in. It's more explicit, and if it needs modification or fixing later, side effects are less likely. That's a pretty easy example though. But it does point out that sometimes it's easy to start seeing regexes as the answer to everything. Things like this can help readability too:

if (fooString.IndexOf("something") > 0)
fooString.regexReplace(somestuff)
else
fooString.regexReplace(otherstuff)

Sometimes doing lots of ors/ands in a regex can get totally hairy, so splitting them out into simpler expressions works well too (and helps show unit test coverage).

And finally, the links to tons and tons of info abour regular expressions.

Regular-Expressions.info - Covers everything very detailed.

Good discussion of performance - This is good for it's explanation of exponential running time issues:

Something like "(.*.*)*x" in a string not containing "x" can take exponential time, n^n, so a hundred characters of text could potentially take years to match unless an early match is found. The inner term ".*.*" takes n^2 because of the multiplicative effects of concatenation operator. (I used two wildcards just in case "(.*)*" is optimized away.) The outer wildcard compounds the running time by a power of n.

May 19, 2006

AAAGHGH. I hate assembly loading in .NET. The whole system is designed for shipping software, and all the great advice out there is from the point of view of shipping software.

So what if you're running a web application, where you completely control the hardware and software, and there's almost no need for versioning, and the act of trying to make strongly named assemblies go properly is a total nightmare? Oh, you're screwed then. You will have nightmares that put the worst java classloader problem you've ever seen look like a nice walk in the park on a warm spring day.

And yes, I'm even including the time that the people before us had tossed nearly everything into the system classpath, had each EJB configured as it's own application so it had it's own classloader, some classes -were- required to be in the system classpath (Weblogic custom realm) but they used some objects from the EJB and web tier, web-layer objects (like HttpSession) were being passed into session EJB's as arguments so servlet code had to be available to EJBs, and we had to unravel the whole thing to be able to have our sandboxes able to update without restarting weblogic.

Walk in the park compared to what we're going through right now. Load() vs. LoadFrom() context. Way too much magic.

Update: Remembered some more stuff about why that java situation was still a pain, but it was still conceptually easier to understand what was going on than this issue is.

May 4, 2006

Software is hard to build because there are so many approaches that will lead to less undesirable results down the road, and so very few that lead to success.

It's like a 3,000 page choose-your-own-adventure book that only has one positive outcome in the hundreds of possible endings.

April 30, 2006

Where have I been? Debugging a gnarly memory leak in our just-released production application (not in our code, but in another part of the system).

I wouldn't have even thought it possible to leak 2-3MB of memory per request. Yes, that's not a typo, that's per request. How? Through a combination of an xsl transform retrieval method that the caller thought was caching underneath but wasn't, some old untouched xsl, and some weird behavior with script blocks in proprietary script blocks from MS in XSL files.

So we were leaking assemblies like a sieve. Thirty or so per request, as seen by the perfmon counters for bytes in loader heap and current assemblies (under CLR loading and CLR memory).

Remember to always check your bytes in loader heap when you have a CLR memory leak, as well as the normal heaps. Otherwise you won't see the growth, but you'll see the private bytes growing, and assume that it's a native leak.

I also found it interesting that the CLR has 3 generations instead of Java's 2. It makes watching for heap growth in the aggregate counter (# of bytes in all heaps) a lot more difficult. I recommend looking at the behavior of each heap individually, it makes the behavior a lot clearer.

Anyway, problem found. We may have a slow leak as well, but first up was definitely to stop this bleeding.

April 17, 2006

Gregor starts with some good explanations of what makes code readable in March is Not a Number., using examples from Eric Evans.

I'd add that when doing this sort of coding, a definite design smell is when you start mixing domain specific languages. It's likely in this hacked together web world that you really have several domain specific languages. Somewhere you're logging a user in with cookies, calling a web service to verify someone's identity, making some business decisions, or retriving a customer profile via an MQ message. All of these layers form their own little domain specific languages.

So how cleanly these languages are separated I think makes a good canary for how readable your code is. The code has readability problems if you're pulling a value directly from the session, checking against something it in your domain model, then directly calling a generated web service proxy and catching soap/http exceptions.

A business person doesn't understand (necessarily) http, or the concept of a session. If you really have a domain driven design, then someone who understands the web should be able to look at your web layer and get what you're doing with cookies and the session. The business person can look at the core logic and not see implementation related cruft. And the back-end code can be understood by server side developers who understand web services and all the things that might go wrong while calling them.

The more seperated your concerns are, the more readable your code probably is. I do hope you have good refactoring/code navigation tools, because sometimes this stye can get a little deeply nested. I don't think that's a problem; I prefer it that way.

April 11, 2006

I think this piece from Hacknot, "In praise of Code Reviews" hits upon some of good reasons people don't do code reviews. It do think there are a few more though.

Tool support. It would be really, really nice if you could check something into a holding area for review before it was actually committed. I know some source control systems can handle that, but it's usually non trivial to set up, or hard to use. I'm not in favor of the "SCM as a workflow tool," but I've always thought that a place to stash things for review that was easily integrated would come in handy. Not an excuse, but if tool support were better they would be more common I think.

More importantly though, very few people can do them properly. Every mandatory code review system I've ever seen produces a bunch of people talking about formatting, or naming conventions, etc. Stuff that could practically be taken care of by a tool is written up for fixes. "Missing standard copyright header." "Not throwing standard exception type here."

I've seen some of the worst, buggiest code I've ever seen fly through code reviews with a "looks good." In fact, of all the large codebases I've seen, the worst have all passed mandatory code reviews. Why? Because it's hard to criticize someone else's code. People take things personally. And in the interests of not causing divisions in teams where people maybe don't get along so well already, even those who know better have a tendency to keep quiet.

The only fix I can suggest if you want to start reviews is to start with your own code, and get someone on the team who you have a good relationship to start really digging and finding defects. Set the tone yourself, and hopefully a culture of constructive criticism will work its way into your team.

If you already have that culture ingrained, then I'm guessing you're already writing less buggy code than teams that are afraid to call out bad code when they see it. If you don't have that culture, team dynamic, or type of people amenable to it, you need to develop it. Otherwise no code review is going to help very much.

March 24, 2006

John Siracusa asserts that Apple has a lot of work to do to avoid a Copland style disaster.

His fundamental assertion is that managed API's will dominate in the future, because eventually they will be fast enough.

My argument is not that people have been saying that for years and that it's never "happened." While true, that's not the whole story. The real reason I don't think managed API's will take off is because the most heavily used, popular, and (most importantly) money-making applications are implemented in non-managed languages.

These applications have huge established codebases that represent a large investment. And it's very likely that low-level code will always be faster. So even though a newcomer may be able to add features at a faster rate (unlikely BTW, since the established vendor has a lot more money, manpower, experience, and mind-share), the established app is still going to be faster. For every managed app, there is always going to be a company willing to spend more in development, and produce a non-managed version that's faster.

Not to say it's not important. A lot of smaller, innovative apps will crop up, and they are going to want to go with whatever platform provides them the least amount of fuss. The difference between the above article and my point of view is that I think Objective-C with GC is good enough to get there. I'd say that most of the smaller, more innovative apps are showing up on Cocoa. Adding GC helps a bit more. Eventually morphing Objective-C into having a higher level, more managed option with the same programming paradigm could push things even further.

He's right about one thing though, Apple shouldn't rest on it's laurels. Enticing developers to the platform is about the most important thing they can be doing.

March 22, 2006

Ran into a few interesting defects that quickly turned our codebase into a C# trivia contest this week.

We have a value object in our domain that three possible values, and they were being stored as readonly references in static variables. So of course the references are readonly, but the objects weren't (because we are persisting with NHibernate which seems to be requiring read/write parameters).

So, if you change something in a unit test (which I'm not sure why it happened anyway), be careful not to change one of your permanent copies, lest you end up with unit tests that break in the suite but work fine individually. Yuck.

Lesson two was about distribution. Reference equality looks nice, but is no fun when serializing your objects, since your object coming from a deserialization isn't going to be the same object as the static readonly private reference. Whoops again. So implement .Equals() and remember to use it.

Better designs to solve #1 we're thinking about including a read-only subclass that at least throws exceptions when it changes that we'll assign instances out as. #2 is less fun, because I don't want to override ==, since I strongly believe that should stick to it's meaning. We'll see if an intense education campaign can keep us out of trouble.

March 6, 2006

Wil Shipley wrote up a story about the tracking down and fixing of a tricky to reproduce bug. It's a good example of how good debuggers/troubleshooters think and find problems. That's something always always tough to put into words, so any time anyone gets close I like to point it out.

For me, the tools used aren't quite as step-by-step. I tend to look at all the info up front (stacktrace/exception/defect report) so it's all sort of in my head when I start, which sometimes helps catch the part that doesn't make sense a little earlier.

If Wil's steps hadn't worked, the next think I would have tried would have been adding temporary assertions to verify some of those assumptions (even assumptions that I just looked up in the documentation). Sometimes the docs are wrong or confusing, and the best way to find out what's really going on is, well, to find out what's going on.

I often find that doing that finds the problem, or perturbs the bug in a way that gives more clues. Don't be afraid to experiment; sometimes the same defect occurring in a slightly different circumstance can give a lot of information about it.

February 13, 2006

Just in case anyone is curious, if you have some gnarly, 500 line long method with 7-8 levels of nested if/then/try/catch/for/while blocks, then hiding 25 line chunks that are the last 2-3 nested block levels behind a c# region is not generally considered refactoring.

February 9, 2006

One of my favorite analogies in building systems is "Moving Parts." Things that are simple have few Moving Parts, complicated things have more. The more moving parts, the more likely something is to fail.

But I don't think the notion for me is tied only to some absolute measure of complexity. The Weblogic proxy plug-in for example, is certainly complicated. However, it doesn't have a lot of configuration, a lot of dependencies, or maintain a lot of state. And so even though the code may be complicated, the component itself is very reliable (I've attributed very few problems to this plug-in beyond initial configuration).

Moving parts are stuff that -do- things. It's the data in motion, and how the parts of the data flow are connected. I think this idea needs more fleshing out.

If you need a data file from somewhere, then having the data pulled from a DB to a file system via a cron job, then having another cron job ftp it to the remote system, then some process on the remote system that runs every 15 minutes notices the file, imports it into a local DB, then another process generates an XML file and pushes it to a server where it's remotely loaded by the box that needs the feed...that's a lot of moving parts.

Moving parts means more things likely to fail.

In code, moving parts are hard to test. They require specific environments, and criteria. Maybe you can mock up some of the interfaces, maybe others you can't. So that tends even more to make them less reliable, because they aren't nearly as fun to test as clean, isolated components.

Think about moving parts as you design; state, timings, external dependencies...those can all be moving parts. The more you can minimize those, the more reliable things will be.

January 25, 2006

Objective-C seems pretty decent, it's fun to write stuff on a Mac, and I plan on continuing. The biggest annoyance is the total lack of refactoring tools for Objective-C in XCode. It's even more of a pain then in Java because changing method signatures requires touching two files. Yes, you get a compiler warning, but that's no fun when you're expecting your program to run and see the lastest-greatest result.

It's definitely something I miss, although I do appreciate why it's difficult. Does anyone know of any tools that might help? I'm tempted to see if XCode has a plug-in architecture (which I assume it does)...

December 20, 2005

Dave Churchville writes about J2EE killing OOP, and I think I mostly agree. Certainly a whole lot of the applications I've seen really aren't very OO.

For me the amusing part is that truth be told my own stuff hasn't been that OO. It wasn't really until this latest project, where we had a clear mentor who knew what he was doing, do I think the idea finally clicked home for me. I always had a vague notion that objects with a giant pile of getters/setters and not much else weren't really the best use of OOP.

But put me (and the rest of the team) in front of someone who really knows what they are doing 5 days a week, and very quickly it all starts to click.

It goes back to mentorship again. Teach people to design OO applications and they will. Teach them to stick stuff into the "web layer", "business layer", and "data layer" then translate it 50 times between each with DTO's and proxy classes and they will.

I had to start working on an ASP.net application, where we treated the code behinds and pages as completely dumb dialogs and drove them with controllers before I really saw how to separate the behavior out of the http infected classes.

Anyway, I can't agree more with this part:

But I fear that the last 5 years has produced a pile of brittle, over-engineered applications whose internals are exposed.

Most web-apps definitely suck, and it takes a long time to fix them once they are broken.

Update: One more thing. Corporations want it this way, because it matches with the way they see the world (if we buy a good enough tool/create a good enough process/etc. then we will be able to create good software). Getting OO right is hard; a lot harder than just handing someone a bunch of framework classes and telling them to put the code in between the /* your code goes here */ generated comments. Corporations don't tend to trust the "hire smart people and let them take care of things" model, so they demand tools like this. Because Microsoft and Sun want to stay in business, they produce them.

Remember, software companies exist only to sell you software. If you're successful, great, maybe it means you'll buy more software, but the real quality of your end application I think matters very little to them. The people writing the code matter very little, because they are not the ones generally making the decisions about what platforms to use. So the platforms are designed for what the managers would like reality to be or how they see it, regardless of how things really are.

December 19, 2005

So right now, we're suggesting removing persistence (at least the user visible kind), which is implemented rather oddly in our app, while we make some major data and object model changes that are going to be required by some new desired functionality.

If we leave in persistence while we do this conversion, there are a ton of things that will crop up that will make the conversion take longer, because we'll always have to sync the data model with the object model, and we have a lot of environments to deal with as well. So we can go a lot faster without worrying about persistence (plus, all the business rules need to be evaluated in the context of variable times between each steps if saving is allowed, greatly increasing the edge cases, analysis, and QA effort).

So we yank it, so we can go faster. Even if we put it in later (once the sweeping changes are done), the overall time will be less than just slogging through. Interesting idea; temporarily removing difficult functionality to get through a rough patch, then put it back in when things have stabilized. We're sure we can get more done in a year by doing it this way, rather than the slog-through-it approach.

So I see this as analogous to racing lines. The basic idea is that you slow down early in a turn when racing a track. The goal of course is to minimize the timewhen you are not flooring the gas pedal. So you slow down early to get slow enough to get around the turn, then floor it again.

This way you minimize the time lost, because when you come out of the curve, you are going faster, and the straightaways are longer than the curves.

Kind of the same thing; taking a small time hit on a small part of the course to maximize your performance across the whole course. Global maximums vs. local maximums. Tom (Thoughtworks PM I know) would be proud.

December 6, 2005

Here are three reasons to use the GAC.

(For those of you not familiar, the "GAC" is Windows' global assembly cache; a place to put .NET dlls where all apps can get at them, so you only need to load one copy and different versions can be requested)

When you have a large team that works in a distributed fashion, on a bunch of web applications that share some common code and hardware, I don't think the GAC buys you much. In the above, #3 is applicable. However, if the code sharing is small, even if you have 10 copies of a 50k dll, is 1/2 MB memory savings really worth the extra pain?

The pain of either forcing people to migrate at once, or support hard-coded version numbers, or have policies all over the place is non-trivial. The versioning can become a real nightmare. What if you want just one application to use a newer version of the library? It's just a lot easier to manage if the whole app can be xcopy deployed to one place, and if you need to fix its copy, you can.

The GAC is great for commercial software, but it's quite the pain for my purposes. Just because you share code, does not mean that you have to share -instances- of that code. As long as you're not cut and pasting that shared code, using someone else's binary dll is perfectly legit. If you have a thousand web apps, the story might be different.

In fact, versioning all together is a pain for web apps that you control the deployment of. Stick everything at 1.0 and forget about it. Life's too short to have to rebuild just to update some stupid assembly reference when you know you are deploying the right version anyway. If you want a version number, but it in another field in the DLL metadata, but leave the .NET version at 1.0.

December 5, 2005

So it strikes me that the way we are generally expected to build software is just plain wrong. It's not just "Agile" vs. "Waterfall", but it's the entire context and methods around construction.

There are too many analogies to engineering, or architecture, or physical construction that are ingrained in the heads of so many, but these analogies are just models. Models of facets of software that do not encompass or define the entire activity.

Software is messy. There are little or no real world constraints. It has an artistic quality to it that's subjective; one person's elegant solution is another's nasty hack. It can't be codified into a few simple rules that anyone can follow.

Software development is a unique activity, a combination of creativity, problem solving, art, communication, and learning that the world has not seen before. Embrace that, and focus on what's important, rather than trying to fit software into a box that looks a lot like engineering or building construction.

November 28, 2005

Signal to Noise (37 signal's weblog) has a posting called Citibank Horrors: How hard is everyday banking? - Signal vs. Noise (by 37signals).

As someone building software in this space, I can vouch for the difficulty. While not quite rocket science, it certainly has it's share of unique constraints. Too often these constraints are self imposed; legacy systems that don't communicate, entrenched political domains that call the shots. All that's left is for some teams to try and isolate the customer as much as possible from the pain (think anti-corruption layer in domain driven design terms), but even that isn't always doable.

Of course, the solution is to replace or update these legacy systems, but that involves lots of risk. And most financial institutions (probably for good reason) are risk adverse. So you get that bad customer experience as a result.

October 31, 2005

Mark Russinovich found some strange, malicious software on his system and did some digging to analyze and find the source. Turns out it's Sony's DRM software, IMO illegally installed on his system. So I find it really absurd that under the DMCA, I bet they might consider coming after him for "illegally" reverse engineering their software and posting the details.

The world is truly upside down.

For those of you that don't wade through the whole article the basics are that Sony is, without the user's permission, installing software that hooks into the very lowest levels of the operating system (where you must be a really good programmer to keep from breaking things and as a rule, most professional developers are not really good programmers.), dilberately evades detection, and gives you no way to uninstall it.

October 24, 2005

Here are all of my photos from OOPSLA 2005. Not a huge amount or variety, I trimmed the totally useless ones. Persue at your leisure. My session by session report/thoughts of OOPSLA is coming soon (target is to finish on the train ride tomorrow morning).

October 20, 2005

It's the last day of OOPSLA, which is good, because I've had about as much talk about objects as I can handle and would like to return to real life at this point.

One thing I'm taking away is that I'd like to write more code. Work of course goes in fits and starts depending on place in the project cycle, and I need to write more code, if for nothing else other than practice or to expiriment with new ideas. I'll be trying to figure out how to better do that.

On Tuesday night we attended Eric Evan's Domain Driven Design "BoF" session. That resulted in a spirited discussion, of which the takeway was pretty much what we at some level knew already. That it's going to be our responsibility to help the business customers understand the back-end model, because it doesn't just have technical limitations, there are real business limitations imposed that have to be taken into account (sadly).

Other than that, we've had great dinners each night; Tues night was Russian-Georgian food at a place called Pomegranate. Wonderful stuff, highly recommended. Last night was Sushi Ota, which was good as well. The Unagi was excellent. It was the same bunch of people both nights: Pravin, Gregor, Yuji (software developer from Japan), Lorin, Aman, and me, augmented by one of Lorin's old high school friends last night. Pravin and I kept the arguments about our relatively useless but heavily embedded middleware to a minimum. In a crowd like that it's okay to talk about software, but still not okay to prattle on about work.

So, it's been a educational time, I'll probably make a post with my notes/thoughts on each session once I think about them a bit.

technorati tag: oopsla

October 18, 2005

Okay, so it's my 2nd day of stuff at OOPSLA. Here are some observations.

Tutorials: As suspected, these do not seem to be where the real action is. Sounds like we scheduled them okay though; not much else going on Sun/Mon. But make sure you pick ones that you don't know -anything- about. I've been to three that were esentially driven around particular books, and if you're planing on reading the book or have read it, the tutorial is probably redundant.

All mine are oriented towards object design, since I consider that to be a weaker point in my toolbox, but I think in aggreate these have all helped a lot. My favorite so far has been Kevin Henney's talk on effective interface design. He had to rush at the end, but was pretty enganging, and there was a lot of profound, thought provoking stuff in the talk.

Last night, we hung out with Gregor Hohpe and went to Region for dinner, which was really tasty. Gregor's a great guy; he gave us a lot of crap for missing his talk, but it was his misfortune for being scheduled opposite Eric Evan's Domain Driven Design tutorial.

Let's see...we also took a quick trip to the Salk Institute, which I believe is where my friend Rashmi did some graduate work. Lorin (from work) is really interested in architecture, and the building was designed by one of his favorite architects. From the outside, it seemed like a boring goverment building, until you get around to the coastline side, and it's got great teak inlays which contrast with the concrete, all kinds of windows at 45 degree angles facing the water. Pretty nice overally. Here are a few pictures of the wanderings.

technorati tag: oopsla

October 15, 2005

I'm packing up for OOPSLA'05 in San Diego right now. Through the goodwill of my boss and my employer (and the dilligence of Lorin in bugging my boss and employer), we get to hit it this year.

This is my first confrence, I'll try to post back on the interesting things I pick up while there. I think we probably overdid it on tutorials; I'm not sure what the usual recommendations are (if you have any confrence scheduling recommendations, please feel free to comment here). I think I left a day or two totally free of tutorials, hopefully they are the interesting days (I probably scheduled stuff right over the keynote or something, probably a classic rookie move).

Anyway, watch this space for photos and reports; I have a lot lined up on object design, because that's where I think I need the most improvement.

technorati tag: oopsla

September 28, 2005

Wow, so that was quite the annoying ride getting Ruby on Rails running. Through about a thousand posts, I finally stumbled into the right incantations to make it go. There seems to be a whole bunch of suggestions/tips out there, with very little explanation of what they do. I thought I'd try clearing some of that up by walking through the individual problems I had, and putting best guesses down as to the cause. Thanks a ton to Rufy.com's scripts for fixing some things. Without further fanfare and in order of appearance:



"No database selected: SHOW FIELDS FROM recipes"



This is because your mysql driver in Ruby is no good. You need to get a version that works with your version of mysql. This is a best guess; to fix, go grab the most recent from www.tmtm.org (I used to 2.7)



When tryiing to build the libraries, yo get:

checking for mysql_query() in -lmysqlclient... no

checking for main() in -lm... no

checking for mysql_query() in -lmysqlclient... no

checking for main() in -lz... no

checking for mysql_query() in -lmysqlclient... no

checking for main() in -lsocket... no

checking for mysql_query() in -lmysqlclient... no

checking for main() in -lnsl... no

checking for mysql_query() in -lmysqlclient... no

mysql.c: In function 'query':

mysql.c:635: error: invalid storage class for function 'res_free'

lipo: can't figure out the architecture type of: /var/tmp//cc7tIymK.out

make: *** [mysql.o] Error 255

gcc: cannot read specs file for arch `i386'

make: *** [mysql.o] Error 1

dyld: NSLinkModule() error
dyld: Symbol not found: _sprintf$LDBLStub
  Referenced from: /usr/lib/ruby/site_ruby/1.8/powerpc-darwin8.0/mysql.bundle
  Expected in: flat namespace

Trace/BPT trap

September 8, 2005

My friend Tom is blogging now, and has an excellent post about how software development breaks down when avoiding blame becomes more important than producing quality work. Recommended reading. I have to be careful what I talk about with him on the train now so he doesn't steal my few good ideas. ;-)

August 29, 2005

SteveJS asks about our favorite bug stores in Babble : Beeping Robots and Bug Stories.

This one may not be my favorite, but I did lose a week of my life to it. It was a java/weblogic web application. There was a minor problem in accepting data because one of the columns on the database was too small. So there was a sql script written to fix that and prepared. In testing, which was just a formaily because after-all, what could go wrong with a little sql column expansion? One of the testers reported that the thing was hanging on them after clicking around for a while. They couldn't replicate the problem in other environments without the patch.

And yes, their browser would be hanging, because the app itself was crashing. The whole java VM; coredump. But they couldn't supply an exact series of steps. I was able to replicate it very rarely. Very rarely (after hours of submitting loan apps, it would go).

Analyzing the core with GDB just returned the mangled c++ names of the java compiler, and the problem was in JITed code anyway. There was no way to tie that back to a Java method (this is why I -love- the new JDK 1.5 tools). Sun had us install the Sun debugger, but it didn't give us much more. I tried to set the values necessary to produce a java thread dump on shutdown, but it wasn't working.

After 4-5 of days of missteps and bad bug reporting, I finally found a reliable replication scenario. It was frustrating, because it wasn't something out of the ordinary that wasn't even used by the customers yet (a file upload feature), and wasn't supposed to be a test that the tester executed (and swore that they hand't been doing this, so we hadn't checked it). When you uploaded a file, the -next- application you submitted would cause the crash. The code didn't show anything totally obvious; it was a struts app and this was a commons-file upload. Looked okay at a first glance.

So with a replication scenario in-hand, I set to work getting a thread dump. I wrote shell scripts to kill -3 the VM (which generates a thread dump) in infinite loop, with varying delays, trying to get a snapshot at just the right time, but no luck.

So finally with everthing up and running locally with a debugger, I walked through the replication and had my root cause.

We had suspected this method before; it was a debugging call for a stupid log message that walked an object tree, printing out the values recursively. Of course, endless loops are a common VM source, so we had checked it carefully. But it wasn't endless; it checked to make sure it didn't revist identical references (don't ask why code like this was even allowed in just for debugging). So why did succeed when you hadn't just uploaded a file?

Turns out that when you walked the object tree (it was a struts form) normally, it was small and finished just fine. However, after uploading a file, one of the things the form now contained a reference too was the weblogic request...which contains a reference to the weblogic Mbean config root. Which contains some ridiculously large number of objects.

So after file uploading, this stupid debug statement would recursively call itself thousands of times ToString()ing a huge object graph, run out of stack, and crash the VM.

So bye bye a week of my life, but at least we were able to push the needed fix (lengthening a SQL column) to prod.

August 24, 2005

Pragmatic Dave highlights a new book and comments on it a bit in his "My Job Went Upmarket" posting. To make yourself less of a replaceable coding-cog, he suggests lunch with the business and learning more about the business. Those are great suggestions.

What's more though, you might need to re-evaluate the way you build software as well. It's not just understanding the business, but ensuring that you are delivering what they need. Working more closely with them every day, giving them more visibility into what you're doing, and being more honest with them about things are all just as important if not moreso.

It all boils down to the fact that if you're not delighting your customers, you are going to lose your job, just like in the marketplace. Most IT departments and many developers I've seen make it a point of pride to annoy the business. When the business climate changes and a last minute change is needed to requirements, IT departments delight in making the biz people jump through hoops filling out extra paperwork and looking for every excuse to say no.

You may succeed in "pushing back" this time, but it's not healthy. Next project, you might not still be around. Clearly a poor choice of what to optimize.

Note that agile software development, whatever your criticisms of it, provides excellent protection against outsourcing; it's difficult for a team thousands of miles and several time zones away to be as responsive to the needs of the business as a good agile team, even given people of equal skill.

August 17, 2005

Inheritance is generally tricky to get right, and we were looking at a fairly complex base class that needed some modifications. Try as we might, neither me nor my pairing partner could figure out how to attack it.

In the end it should have been obvious really. The abstract base class was doing way too much. The concrete classes seemed similar (they were all marketing "landing pages"), but the behaviors were different. The base class was sort of controlling them, with the intention that the concrete classes would just implement the various validation methods appropriate to themselves.

The problem was that based on the results of those validations, there were different actions to be taken. So we ended up with sort of an inversion of the inheritence, where the logic was all in the base class and it was just controlled by essentially flags in the concrete classes. Not fresh, and a pain to figure out when you're looking at some logic and you have to consider what the effects on all the various concrete classes when you need to change the flow.

So if you find yourself looking at a base class with lots of code and classes extending that that aren't adding a lot of their own behavior, that's probably a warning sign (like I said, should have been obvious but it's easy to get caught up in these things).

July 30, 2005

So I've been a little hesitant about Greasemonkey in general just because of the potential for hassle (always having to change scripts as pages change, security issues, etc.).

Well, I thought of a killer use for it when developing web applications; using it to pre-populate forms with valid data. Create a little script that fills your form with data. If you're working on things like, oh, say 10 page loan applications, filling out all that data by hand when you're trying to test or troubleshoot somtehing is a pain.

I've written up a script for the app I'm working on right now, and it rocks (just adds a little "click to populate" button on the upper left hand corner of the page). It could be extended to create random data based on validation rules, etc..

Anyway, if you're building a web app with a long form, consider some greasemonkey action to speed along data entry, it's saved me tons of time already.

(And in the process, I discovered that Greasemonkey is pretty fun tool for its more intended purposes as well).

July 20, 2005

Last weekend I discovered folkore.org, a facinating pile of stories (I think I've finished them all now) about the original Macintosh design team. It's incredibly interesting reading; how does a team of brilliant people come together for a purpose, and what eventually drives them apart? The variables required to create and maintain such a team are very hard to juggle; if you can do it reliably, then that's a real talent.

July 9, 2005

So on my most recent project at work, I've been involved with very smart, well known external contractors who were brought in to help us out. I've always known what TDD is, the high level of how to do it, and the value of unit tests, but none of those things have ever really made it into any project I've been involved in a significant way.

All that has changed now. I've learned a ton watching and participating while things get built that way. It's a powerful concept. It's also a concept that you can't really grasp on your own, without practice (which should have been obvious to me). I can say that the shift between "I see the value but never have time to try it or do it" and "I'll never program any other way" is incredible subtle. It's seeing the difference between unit tests that don't do anything but excercise the most basic, obvious functionality and unit tests that actually expose the business logic.

I've read all the right books, websites, and blogs, but until now it never clicked. The codebase we are creating, while not perfect, is much cleaner than anything else I've dealt with before, and the team is flying along.

Anyway, give it a shot. I'll see if later I can't articulate some hints to those who like I was, are on the fence or having trouble really getting the most out of the technique.

June 11, 2005

Ulrich Drepper seems to think that the burden for supporting "irrelevant" architectures and proprietary systems is too great, so paring support to the bone will help the open source community increase adoption.

There is no need to develop on those proprietary or irrelevant systems anymore. It does not further the cause of free software.

That is a very short-sighted view. Corporate America is not going to succumb to the views of a bunch of open source extremists and suddenly through out all their AIX software and hardware, thrilled to be moving to Linux. The open source world has to show value. The best way to do that is incrementally. If apache only ran on linux, would it have nearly the adoption it does? No way. It would be a niche player instead of one of the dominant ones.

Things like Apache, MySQL, etc. are the gateway drugs for corporations into the open source world. The "all or nothing" view is going to force them right into "nothing."

There is a valid point within this article; that balancing support for older and increasingly obsolete architectures is probably done better by a seperate project of people who are in a position to test and validate these darker corners. I think that's going to get lost in the flames over the the broader calls to action.

May 25, 2005

Should I continue to use moveable type, or should I build my own blog engine? Certainly it must have been done a thousand times, and I have no delusions that I could do a much better job than some of the other offerings out there. However, it would be a good test-bed for various things (ruby on rails maybe?).

I don't know, it's probably not worth the time. I should concentrate on getting my photo database up, running, and out the door.

March 29, 2005

I thought the one advantage of working in a Windows project was going to be Visual Studio. I thought, "Microsoft's been building this for years, it must be heads and shoulders above Eclipse and what I'm used to." I'm wrong. Even when it comes to features, not counting it's whole model of working, it's behind. The auto complete sucks, no on-the-fly incremental bulding, apparently no way I've found yet to turn off it's infurating "outlining" feature by default...I'm totally shocked. It must be better on the C++/COM side of the house which I'm not using. But the C# part of it appears to be far behind what I'm used to in the Java world. Debugging was a little easier to set up than under Eclipse (when talking to Tomcat); I'll give it that.

But then it requires you to run a web server to load your project. And if there's anything wrong with your web server, like permissions or anything, it just has two error messages to choose from. No logs, no nothing. Just "Sorry, couldn't load your project, fuck off."

Have we not heard of "logfiles" before? How about a "More Information" button at least. Instead, it just says that your web server is installed wrong and points at docs for re-installing things. Nevermind the fact that I've gotten that fucking error message 20 times already, and not once has it ever required re-installing anything. Bad permissions, daring to move a project file on the disk, trying to create a 2nd instance of a project for debugging purposes...those are the real errors.

Some of this is no doubt due to my inexperience with the thing, but it seems that in general there are never any useful errors produced anywhere. Would it be so hard to display an error message? It's not "print shop" where a detailed error would freak Grandma out, it's a fucking IDE, for developers, by developers. I think they might be able to handle "Access denied: http://localhost/projectname/project.csproj".

January 23, 2005

There are a lot of tricks out there to hide css from buggy browers. They seem to get employed almost exclusively. Am I missing something obvious as to why you would want to rely on what you're limited into by those hacks and their general infathomability, rather than just sniffing the user agent and serving up the appropriate CSS? This is even something painfully easy to re-use...and it could support overrides, includes, or even full blown templating. So without a huge amount of maintaining multiple style sheets, you could serve up exactly the css to each browser that you wanted. Even generally allow the user to choose their own css or override the default setting...

Seems to me to be a no-brainer.

In case you haven't noticed, my new project is going to involve a lot of client side work....