Sysadmin Field Notes - Geeky Indulgences Archives

April 9, 2008

I don't know why this just popped into my head, I've thought about before but apparently never mentioned it. The Nyquist limit is most accurately described on wikipedia, but here's a simple way to visualize it that I've always used to explain it.

Imagine a video camera, shooting 30 frames per second. You walk into the frame, and your actions are very accurately recorded. Now slow down the camera to 1 frame every 60 seconds. The camera takes a picture, then you run through the frame in 2 seconds. 60 seconds later, it takes a picture and it's like you were never there, even though you were. You were faster than the sampling rate of the camera, so it didn't capture you at all. It -can't- if you continue to move that fast. You have to slow down, or speed up the camera. That's why the hard limit.

Now, you if you vary your speed, you can play with the frequency of how often you show up. So let's say the camera it still running at 1 frame a minute, and your pacing back and forth, spending 5 seconds in the frame out of every 45 seconds. So your real rate of passing the camera is once every 45 seconds. So the camera starts, as do you, and your first pass comes at 45 seconds and you're gone. The camera fires, and there's no you. You pass by again at 1m:30s...then 2m:15s...then at 3m. The camera is firing then because it fires at 0s, 1m, 2m, etc...So you're caught on film. You're next pass is 3m:45s, then 4m:30s, then 5m:15s, and 6m, and you're caught on film again at the 6 minute mark.

So you're real rate of passing by is every 45 seconds. But according to the film, you're only passing by every 3 minutes. That's aliasing. Because the sampling rate isn't fast enough to capture your frequency, you are "aliasing" to a different frequency. From 45s to 3m. You can imagine when capturing music with samples that this would do baaaad things to the soundwaves, which is why digital audio systems filter out high-frequency sounds somewhere around the Nyquist limit, so they don't alias to the wrong frequencies and destroy everything.

This explanation isn't perfect of course, but I have found it useful to explain the concept.

December 15, 2007

Everyone knows about this tip for un-drm'ing itunes files.

The bonus side effect is that you get the most boring movie ever made. Picture a bad snapshot of the interior of SFO with Hendrix's star spangled banner playing over it.

True art, right there.

July 3, 2007

Okay, last post on the iPhone for a while. Lots of people have asked about the keyboard, so here's a quick video I made.

July 2, 2007

So, a less obvious piece of the iPhones UI is the headphone set. The pause/play button is well thought out. Satisfyingly tactile, you can start or stop music, answer calls, and skip tracks without ever digging the phone put of your pocket. Its actually a significant bump over the iPod, where when the thing was buried in my pocket I found the easiest way to pause was to just pull the headphones out otherwise the routine was "fish from pocket, slide hold switch, hit pause". Now its just "squeeze button."

Cool. And yes this posted via iPhone, on the edge network, from the train. It won't win any download contests but its fast enough for me.

July 1, 2007

Wow. It's hard to not say much about this thing without sounding like a fanboy. Boing boing's iPhone post is spot on for me, and I could some up the whole experience in a single sentence:

Suddenly every other piece of technology I own seems like a pain to use.

The interface has amazing attention to detail; the touchscreen is pixel-by-pixel perfect. Rather than repeat everyone else's reviews, here are a few of my favorite things that I haven't seen much posted about:

Filling out forms in Safari. Works amazingly well, even long ones. There are "next/previous" buttons that go through the form in tab order, so you just have to pick the first field, safari zooms you into it, and then you can type it in, and hit "next", which zooms to the next field and lets you enter it. Then done pops you back to the page and you can click the 'submit' or whatever button. Fabulous.

Plus, the keyboard is field-context sensitive. If you're on something it guesses is a phone number field, it's just a number keypad. If it's a drop-down list, you get instead of the keyboard, a fully scrollable/flickable list of the contents.

Keyboard/Text Entry Works wonderfully. The big variable is hand size (which is why I'm curious that they only support the horizontal keyboard layout in safari, since it's much easier for big hands). I have long fingers, but they aren't too big. I'm already at or past blackberry typing speed. The auto-correction works great, and even picks up proper names out of your contact list, including names and addresses. Just barrel though typing, and it matters amazingly little if you typo something, it will get fixed. The context sensitive bits are a good touch too. If you're in a URL field, you get "/" and ".com" instead of the spacebar. Nice.

Also nice is that if you hit symbol/number mode, when you hit space, it reverts back to letter mode. I find this saves a lot of time.

Fit and finish. Crazy good, everywhere. The feedback is so immediate you feel completely connected to the thing. It's just so natural. The little magnifying glass for text correction has lens flare, distortion at the edges...which is funny and maybe a little too eye-candy, but it's touches like that that make this thing so damn fun to use.

Activation/Sync. Very nice to have all of your contacts automatically synced without even thinking about it. All my numbers in the phone right from the get-go. Activation for me was totally painless, even with number portability. I did it friday night, and I think I finished in about 15 total minutes.

In syncing, it -also- syncs your safari bookmarks, which is a great touch, and in a real, real move of genius, your auto-complete text as well. Most of the URLS I use are there, so entering common urls is -very- fast.

Issues. There is some sort of DHCP problem in some cases, there's a few threads on
Apple's discussion forum. My fix was manually setting my IP and giving the iPhone a reserved address in my router.

Deleting text is the one slight "clunky" spot in the interface. You just have to backspace it...press and hold works fast enough, but I'd like to be able to notch it up just a smidge. After several characters, it does switch to word at a time deletion, which helps. Also, the email icons were a bit inscuritable to me..the arrow indicating reply/forward wasn't obvious. The fact that things like that stick out shows how well the rest of the interface is put together. No cut and paste will weird some out. Not so much the paste, but the "cut" part is something I miss.

EDGE seems pretty quick to me, but to be fair I had a terrible phone in some terribly slow 1.5G network or something before. Web browsing with it isn't fast, but email seems okay (even up in Apple hill in Placerville where we went on a picnic yesterday), and google Maps is nice.

Oh, and it seems to make a pretty good phone too. Being able to set/enter your voicemail password right on the device, and choose/record your greeting is very cool, as is being able to play your voicemails like a media player. Sure, the out-of-order aspect is cool, but being able to fast-forward or continuously replay a particular part of a mail is really the neat part.

And to think this is just a platform. It's all software, and I expect that will be the exciting part over the course of the next year.

Massive props and kudos to everyone at Apple for pulling this off. IMO, the bar for human/computer interaction has just been raised. Sure, this keyboard/mouse might be faster (I was going to to this entry via the iPhone, but the links would have been a bit more than I could bare), but it's about half as fun as well. I -really- want to just be touching my monitor and dragging the windows around right now.

June 29, 2007

I don't see that this can hardly be earth shattering, since we've seen a "ringtones" tab in publicly displayed iTunes before, but the latest 7.3 update includes in it's resources directory a nifty little icon called ringtone.gif.

My question is, was this held out because of contract negotiation type issues, or was it just not quite ready? Luckily ringtones aren't a big deal for me, but it definitely feels a little evil to force to you to pay for your songs that you've already purchased just to use them on your phone. Clearly though ATT is going to want to protect that revenue stream.

June 19, 2007

From cool tools, my favorite alarm clock. We got one of these a few months ago and the usability is way above and beyond most clocks. Seperate adjustment dials on the sides for each alarm time, color coded weekend/weekdays, and being able to see all the alarms at once without a mode-switch.

Best I've seen so far.

January 26, 2007

Raymond Chen suffers from the same disease that I do apparently, just like when I lost my lens cap while taking the pictures to demonstrate my disorganized-ness.

I'm getting better though; expect a new organizational sum-up post in a couple of days.

January 23, 2007

If you're not listening to Bill Withers, why not?

April 12, 2006

A comment on

Boot Camp is smoke... over at the Unofficial Apple Weblog says this:

That might just be the smartest thing I've read about this whole Boot Camp deal. Personally dual-booting isn't what I'm after; I'd be much more likely to use virtualization or even better, true emulation, Rosetta-style.

I've heard this from several people, and I just keep thinking of OS/2, and how it seamlessly ran Windows applications right next to OS/2 apps. In fact, it did it pretty much better than Windows. Writing an OS/2 native app just didn't provide any value. It wasn't better enough. Perhaps it might have kept OS/2 around longer, until their agreement expired and they no longer had enough access to windows to keep the win32 api compatible (remember Open32?).

Virtualization doesn't suffer from that problem, but still. If Photoshop for windows ran smoothly and flawlessly on OSX, how long would Adobe maintain the OSX version?

Virtualization as we know it today is pretty much as close as Apple can get to running windows apps smoothly without cannibalizing some of their pillar applications. It may even be too far; I'm not convinced we'll see it from Apple or included in the OS. Boot camp solves many people's problem who are switching (I don't want to abandon my software yet, I want to play my games, if it doesn't work out I don't want to have wasted $2000, etc...), but isn't seamless enough for companies with large OS X application codebases to just say "run the windows version."

If someone wants virtualization (as many do), there will be ample third parties, and just the extra overhead of purchasing the virtualization software and windows will keep the Mac mainstream preferring OS X over windows apps.

Boot Camp gives people switching a safety net, and that will give them a few more users (just as Daring Fireball's comments on Windows as the new classic had to say). That's it's real purpose I think; removal of one more barrier to switching.

February 22, 2006

From Engadet, and others Apple is announcing "fun new products" next Tuesday.

Lots of speculation about iBooks and such. Maybe, but look @ the invite. Fun to me implies non-major. And the icon is the iCal icon.

So put my guess down as some software updates (including iCal); fun games/widgets/etc. to ship w/ new macs, and possibly lower end stuff, like a new mini or iBook.

April 12, 2005

WS-* vs. REST, python type [via: waxy].

There are a bunch of others if you read the thread, but that one's my favorite.

March 30, 2005

Don't like it that ASP.NET dumps tons and tons of encoded state information into your pages? Just override some methods and compress it. I guess what I'm missing is why you'd use it at all for any complicated application. You gain some up front productivity for sure, but at the cost of having no control over your HTML, making cross browser work a royal pain. More importantly there's no transparency here, so if one of those controls ever stops working, say it posts on the "onchange" event but the server doesn't do anything about it, good luck debugging

Am I missing some major advantage to the postback/viewstate/event driven model? For my money it's great for visual basic where everything is controlled and you don't have the client side/server side separation. But that's inherent to the web. It can't be easily abstracted behind some simplistic browser sniffing and a couple of lines of javascript (how well does that viewstate form work on a screen reader or a mobile device?) If you commit to the viewstate model, you've tied your application forever to whatever Microsoft deems fit to provide you, and whatever browser technology they deem fit to support. There's certainly no competitive advantage there. And likely, it's going to make it difficult to even "hand implement" a few key features that are important to your application. Once you've added about 30-40 forms to your application, you're going to find that what you really needed was something at a higher level, to manage the the interaction of the business logic with the presentation layer, and that code behind pages really don't cut it.

From what I know, this goes for Java Server faces as well. If anyone can enlighten me as to what the draw is, that would be great...seems like trading some short term gains in for lots of pain down the road.

March 25, 2005

I'd like to nominate this gem as the ugliest workaround of the year:

2) For a RadioButtonList The HTML is created for us. We cannot insert extra tags between the radio button and its label. However, we can use client code to insert these tags after the HTML arrives at the client. The following code demonstrates this. This code can be added directly to the HTML, or you can use Page.RegisterStartupScript.

<script>
var s;
s = document.all("RadioButtonList1").item(0).outerHTML;
s = s.replace(/<LABEL/gi,"</td><td><LABEL");
document.all("RadioButtonList1").item(0).outerHTML = s;
</script>

Wow. How about we just do away with the whole "drag and drop"/event driven/vb6 model for serious web applications that actually want to be maintainable instead?

March 25, 2005

vbbox.plan: The transmission tax weblog is a funny little made up story to illustrate the foolishness of forcing Microsoft to remove parts of Windows.

Funny thing is, I almost agree. Except that Microsoft, via it's predatory practices, drove everyone else out. Yes, their competitors made some calls. Maybe they wouldn't have survived on their own merits. But we'll never know, because Microsoft leveraged its monopoly to drive them out of business. Predatory pricing, arm-twisting licensing agreements, etc... All normal in day to day business, so it's not like I necessarily fault MS for it. However, the practices were deemed pretty much illegal, and it was only through heavy wrangling and lobbying that MS escaped with just a wrist slap.

Business is business, may the best company win, no problem. But for God's sake, don't -whine- when the occasional court case goes against you and you have to do something odd to a product, where the courts are just winging it, trying to restore some sort of parity. We call the court's reaction closing the barn door after all the horses have escaped.

Personally, I just think it should have been a fine. A big one. You can't legislate features in software, that's stupid. But you can punish for illegal OEM agreements that block competitors out of the marketplace, even after repeated warnings and adjustments. Big fine; take away the cash reserve and competitors might actually be inclined to enter the market. If Microsoft really is as good as they claim, then they'll quickly rise to the top without illegally leveraging their monopoly, and no one can say anything except "I guess they provided a better product."

I think I've written all this before somewhere before. Oh well.

March 23, 2005

This is beautiful. Recieved today in my spam email box. The html portion of the email includes a virus with some tricks to run itself. The plain-text part that I have mutt set to read if available says:

Get a capable html e-mailer

That's great. Deride my preference for text mail so I can get infected by your virus. No thanks, I'll stay with my "incapable" text-only mail.

Because the world's a much simpler place with text only email.

March 16, 2005

I guess there's a growing flap about VB6 being decomissioned and becoming unsupported. The Mountain of Worthless Information has a good summary (and is where I learned of it).

It all goes back to what I've said time and time again; if you're building important, core business applications, choose a platform that's going to be around a while. If Microsoft is going decommision VB6 after 7 years, then the message they are sending the message that VB shouldn't be used for long term development.

We as programmers/developers/system administrators need to stop playing ball with this upgrade mania. If you're building a website, no big deal, you'll probably completely redesign your site every few years anyway so you can sneak in an upgrade. But if your doing heavy lifiting, find something that's going to last.

Particularly funny for missing the boat is sound off. I agree with some of his assertions, but the parallel to the PDP-11 is just wrong. That's hardware; it wears out. Parts break, become harder to find, and at some point it becomes impossible to use. Software doesn't wear out; in fact, because it works, it gets more valuable as time goes on, and becomes more important to keep it running by whatever means necessary. The longer you can run on it, the cheaper it becomes. The tail end of its lifecycle is where it finally adds value.

What would you write a system in if you wanted it to last for 20 years?

March 10, 2005

So I'm working late, and then Marimba (aka evil peice of shit corporate patch/software management tool) takes over my PC and demands that I have to reboot in 5 minutes. I'm right in the middle of manually merging a crap load of files into the trunk that were left stranded out on the branch, done by someone else on the team. So I have to switch contexts, come out of my flow and save all my work, submit what I've done, mark my progress, etc.

After 10 minutes when the POS finally comes back up, I get started again, and in about 2 minutes it says it has to reboot again. Sigh.

Then I login to bitch about it here, and I've got 55 ping/trackback spams, which is the new thing those aren't protected by the little captchya doohickey, which has otherwise reduced my spam to zero. Since I defiitely don't have the time to deal with that, I'm turning trackback off for a bit.

February 27, 2005

One of my favorite random bloggers makes his IE7: predictions // plasmasturm.org. I agree, thinking that he only left out one thing; a special bevy of client side hooks to make ASP.NET's custom controls work even better with IE. Like expanding the cobbled together event driven model to include support for rich, Google maps type apps. On IE only of course. Other "downlevel" browers will fall back to some very poor HTML only version.


Or maybe this is the domain of some other Longhorn technology already? Anyway, watch for it for sure: "Build rich web apps by clicking and dragging, just like visual basic."

February 20, 2005

Ted Neward makes an interesting obervation, namely that just about every new IT/software thing has claimed enables reuse. That's a good observation, although I disagree with the conclusion. He thinks the problem isn't about enabling reuse. I think that it is, it's just that none of the solutions out there help address the real problem. In fact, I think we're collectively so far off base on really enabling reuse that we don't even exactly understand the problem yet. We certainly know the symptoms, but we're only taking stabs at the dark at ways to fix them. If you re-use, you get more complexity from writing more general code. If you go from scratch or adapt an existing approach, you get complexity from excessive duplication.

So maybe lack of ability to reuse things is really just a symptom of the incredible complexity of all this, which is the real heart of the problem.

February 19, 2005

Putting some thoughts together inspired by things I've read recently.

1. Phishing and more directly hacking local computers is only going to get easier and worse (witness unicode DNS and be afraid).

2. Businesses (financial institutions in particular) are going be increasingly asked to do something about it, either by customer demand or legal action.

So what's a bank to do to protect it's customer? Some European banks have gone with scratch off cards that provide one-time passwords. I thought that was pretty good, but it doesn't solve much if the attack is done right. Attacker steals password, returns error message to user to try again in 24 hours. Attacker uses stolen credentials to log in and steal.

SecurID fobs? Also no real good; attacker steals number and password, and has an average of 30 seconds to log in with that number. If they fail, they can even just ask the user again to keep supplying numbers, returning bogus "try again" errors until they successfully log in.

So what on earth does that leave? Challenge-response tokens? If the attacker acts as an intermediary, again they can snoop the challenge, get the response back, and send it on feeding a fake error message to the user.

None of this bodes very well. It's pretty much impossible to secure a comm. channel when one end of it is fully compromised...ironically enough, we'd need something like Microsoft's Pallidium, with hardware support to support a secure channel for entering passwords. But this is probably too complex to get right.

Assuming that the attackers don't get too fancy in the short term, a stopgap is to give browsers much better SSL interfaces. "You are about to securely visit www.wellsfargo.com; when you last visited the site it was owned by "Wells Fargo Inc." (The CN of the cert). That has changed, and it now claims to be owned by "Some guy". Do you wish to continue? Click here to see the differences between the old certificate and the new certificate."

Of course, assuming evilhackers have complete control over the client system, not very long until they just disable that functionality, or patch the browser to not display various versions of it.

We need much, much, much better client security for average folks. I don't even know where to start. Maybe we should just take all these websites down, call it a good fun several years, and go back to banking in person, at a branch...:-)

February 11, 2005

In the canonical Word weblog, we have The Case of the Missing Tab today. My goodness. No wonder I could never figure out exactly when Word would/would not insert a tab vs. indent the paragraphs.

January 28, 2005

Ugh. For the last time, redirect is not that same as a forward (or server.transfer for ASP folks). It's just not; they have inherently different purpose and effect to the user. So if you're building a product/framework for the web, say, oh, Microsoft's CMS Server, then it should probably work with both. Sadly, no such luck. Without going diving in the internals, there's no supported way to transfer control to a CMS template without an extra client round trip. F'in brilliant.

If you're still mixing up what a redirect is vs. a forward/transfer, go look it up.

Update: Fixed the HTML link and some wording. Man, I get incoherent at night when I'm tired recently...

January 24, 2005

Props to Donald Raab for writing this article, which really entices me to take a real serious look at smalltalk. Now if only I could find a tutorial with as much clarity.

Of course, the last thing I need right now is to pick up another pet project. I'm dealing with gigantic ASP classic/ASP.net hybrid system at work with about 10 different major moving parts, my image database which is crawling along, having to get back into client side html/javascript development for aformentioned work project, and now school for this semester is going to involve writing a bunch of systems progamming in C (interprocess communications mostly).

All this and I'm still not more than 200 pages into The Dark Tower, a book I've been waiting with baited breath for probably 12 years now. There was a time where I would have finished it overnight.

January 21, 2005

Yay. Someone wrote up an excellent article mirroring exactly the reason that xhtml (as commonly applied) generally bothers me. Now I don't have to bother, and his is much, much better and informed than mine would have been.

December 29, 2004

Paul Thurrott's Internet Nexus says:

And that's the point. The HP has all those things and more. Yeah, it's a bit busy looking compared to the PowerBook. But it's also more powerful and more full-featured. It does more of what I--and most people--want. Sure, the Spartan design thing is cute for a little while, but real people need to get work done. And in the end, more should cost more, not less. Conversely, less (the PowerBook) should cost less. But it doesn't. It costs a lot more.

Interesting that the exact same reasoning by Apple (form over function, simplicity over complexity) equals a 70-90% market share in the MP3 market. Do people expect computers to be complex, and become suspicious when they are not?

December 24, 2004

Silly meme: Erik's Weblog : Page 123

Page 123

1. Grab the nearest book.
2. Open the book to page 123.
3. Find the fifth sentence.
4. Post the text of the sentence in your journal along with these instructions.
5. Don't search around and look for the "coolest" book you can find. Do what's actually next to you.

The answer for me is:

"Three dimensional plots for rule base 3"

December 16, 2004

So I might end up trying to help out a project with their build and release process, but it's on Windows. My windows skills have no doubt atrophied a bit, but I do remember hearing about MSBuild. I thought to myself, hey it's been like a year or two since I remember hearing a lot about that, they must have some shipping code by now....

Nope, nothing not in beta. To boot, it's part of VS2005, which presumably isn't schedule until 2005 (at least). To kick extra sand in everyone's face, it doesn't support (and might not by relesae time) .NET framework 1.0 or 1.1. And finally, if it only supports .NET 2.0, it would seem that implies it only supports .NET at all, so are you out of luck for building native win32 stuff with it?

Anyway, I love the idea (of setting up your build once in an IDE and then getting a modifiable file that you can use to kick off automated builds) of the thing, but it's just vapor. Sigh. Hopefully Nant is okay...

Why do I have the strange, creeping suspicion that I'm going to be writing Makefiles again?

Update: Fixed nant link.

December 15, 2004

Weblogic 9.0 Beta is available today (sadly for windows only), and check out it's support for deployment plans. I've only taken a quick look, but it looks like this is the answer to my prayers. You build your app, but then specify what parameters (anywhere in a deployment descriptor or the weblogic configuration itself) that will need/are allowed to be overridden at deployment time. Then sysadmins override those (with a local XML file I think), and it will insert them automatically.

I'm reading this right, this allows your J2EE app to be one EAR, plus one deployment plan XML file. If this works, I'm in heaven.

December 15, 2004

My nomination for the most pointless open source project of the year.

December 5, 2004

This from the New Yorker. It's a long form piece, but even if you've got a short attention span like me I recommend checking it out.

We are used to thinking that a doctor's ability depends mainly on science and skill. The lesson from Minneapolis is that these may be the easiest parts of care. Even doctors with great knowledge and technical skill can have mediocre results; more nebulous factors like aggressiveness and consistency and ingenuity can matter enormously.

There you have it. Team cohesion, focus, drive, rebelliousness when necessary...those are the things that can make the difference between the top of the bell curve and the bottom, even when your people are just as talented as other teams, doing essentially the exact same procedures.

November 27, 2004

From My hovercraft is full of eels: Assistant Orange Peelers:

This kind of "siloing" occurrs all too often in IT shops as well where each person has a specific task: The GUI Guy; Miss Middleware; Database Dude; Build Master; Application Deployer.

The worst effect of too-specific siloing is that no one understands the whole system. When it breaks, it's almost impossible to troubleshoot because you can't do it without 10 people, none of whom share a common vocabulary.

And of course usually some of those resources (usually network or system admins, maybe architecture) are centralized so they come from wildly different groups. Then you get to waste time arguing over who does the tasks that fall into the spaces between silos.

November 24, 2004

The reason YAGNI and DTSTTCPW are gaining in popularity and work in practice is because we don't know anything about what future enhancements will be required. Future enhancements don't care about your architecture, code, or technology; they are orthagonal to all of that. They can cut right across your application's carefully maintained layers. It's quite likely that "premature generalization" will not exactly meet the eventual need anyway, which will require a pretty heft rewrite or worse (and more common), two different subsystems for the same thing living forever in your application.

I think premature generalization is much like premature optimization, but it's not nearly called out as often.

November 23, 2004

Hey, AntFlow is a great idea. I've got all kinds of ideas for using it; we have tons of automated processes that could be automated a lot cleaner with this thing (particularly auto-deploying builds).

November 23, 2004

Read this. That pretty much sums up the heirarchy of production code as I've seen it. Too bad there's precious little "level 3" code (or people capable of writing it) out there.

Level 1 also includes overengineered code. Cargo Cult use of design patterns, mixing up classes and objects (in the form of deep object heirarchies where all the subclasses should really just be instances of a class or two), usually acommpanied by lots of UML...

November 20, 2004

This is cool. A GPL for music. I found this at the DNA Lounge notes.

November 19, 2004

The "Butt Ugly" weblog says that spam's not the big problem, "real" email is. I couldn't agree more. Bogofilter makes short work of spam, but my time and brain is the only thing that will make any headway against the ~3000 messages in my inbox at work waiting for me when I get back to work. Somewhere out there there's a trainable email filter for categorizing emails. I wonder if that could be bent to train it for messages that you consider more important, rather than other categories. I'll have to find that thing...

Update:I was thinking of ifile. There is also dbacl, and others. I think this idea might really have some legs, but I don't think I get enough home email to make it work, and at work we use exchange/outlook, which would make doing this a royal pain.

November 16, 2004

Simon is talking about environmentally-friendly configuration within webapps.

For those of you who know me, This is something that I've long consider to be a mostly-unsung problem in developing web applications. Managing all that configuration information is tough; it's in different formats (regular property files, log4j properties, app-server specific deployment descriptors, app-server specific configuration files, shell scripts), it can be large (some of our applications have a couple of hundred properties that must be managed), and if you work at a big company (this is a big one), then different groups should probably own different parts of the settings. And most distressingly, the vendors provide very little support for handling any of this.

For example, say we're talking about the production database password. It usually goes into the app server configuration file somewhere. Some products will support easily pulling this from somewhere else, some will not. Taking the weblogic view, it's stored in the main config.xml file that contains the configuration for the whole server. Now, this file contains a bunch of stuff that developers care about; application component definitions, JNDI names of resources, etc.. However, the developers have no need (and in some places aren't allowed) to know what the production database password (or IP, etc.) is. So you can't just check it into CVS.

Plus, if the system administrators want to change it, they should be free to, whenever they want, without having to push an entire new build, built from CVS. If it's in CVS and the admins change it locally, then someone needs to propegate it back to CVS so it doesn't get overwritten, but the sysadmins might not have access to CVS, so they have to get a develoepr to do it. In the right property file of course.

There are a host of things like this; anything that's an IP, a username, password, any security keys, etc. should definitely all be managed by the admins. There are a few grey area things, like URL's and tuning settings (thread counts, connection pool sizing) where both the admins and developers probably need to talk to figure out the correct one.

I've rarely seen this separation done well. Most of the time there ends up being either a sysadmin with some development chops building the whole system and maintaining it, or vice-versa. What's really needed is a nice, complete configuration system from the app server vendor that allows you to seperate your config files into multiple parts. So the developers can put whatever they decide they need to manage into CVS (things that should require a build to change), and other stuff can be split out and left on the system itself, persistent from build to build, and managed by the admins (probably via rcs, or cfengine, whatever their normal workflow is.

I've tried to basically do that by wrapping similar functionality over all the config files via vpp and a lot of templates. I think this works the best so far, but there are a few problems with this method (I can detail them out if you're interested).

My other thought is to make people build the app more like shrinkwrap. Pretend you have no idea who's going to use it. Create a from-scratch configuration/install wizard as part of the code (or use an off the shelf one) that configures the app as the admin sets it up (it of course would have to support some sort of automated unattended install file). So when you drop a new build to the admins, it's a fully contained unit that you don't need to worry about configuration. Depending on how vertically siloed your admins are from the developers, this may or may not make sense for you. It does increase the testing burden; but it would be a nice, elegant solution.

November 15, 2004

I've been playing with Spring and Hibernate both at work before I went on vacation and on my on-again-off-again image database (you know, inbetween taking care of a toddler, a recovering wife, homework, a new baby...all that "Free time"). Both are pretty sweet, apart or together. I had finally finished rolling my own mostly trasparent persistence on top of iBatis SQLMaps for the image database; it took a long time, but at least it was finally done and I could continue into actually realzing the real part of the application. Then I decide to rewrite it (again) with Spring, just to learn it. In just the little amount of time I've spent though, things are much futher along. It really didn't take long at all to hammer the persistence out with Hibernate, and most of that was overhead in learning it.


The spring MVC framework is pretty good as well. It doesn't take very long at all to get the basic "CRUD" forms for everything up and running. I'm finally working on some of the trickier, more interestings parts of the thing. I'll be posting a few more things here about it as it expands.


Right now I'm working on mapping one of the more complex forms to the right Spring concepts. Should I just use the domain objects, or does this form need its own command object, and how complex should propertyEditors be? I asked this question on the spring forums, feel free to contribute thoughts or ideas here or there.


Basically, I'm working on the image upload form, which is 3-4 step wizard-type form. Upload either an image or a zip of images. The next page is the thumbnail(s) with fields for the metadata. You submit that, then they are all saved in the database, and the next page is (optionally, you can hit finish at the previous page) is the images with the options for manipulating them (adding borders/rotating/resizing, etc).

October 28, 2004

Joel's having trouble with IIS. This is -exactly- why good admins hate windows. Most of the debugging he's done has only been so focused because he's an experienced win32 developer. Most admins wouldn't even have gotten this far.


Windows just has a lot of hidden complexity (black magic). Which is fine when it works, but when it breaks, look out. Where is "truss"? Where are the detailed logs? Where is the detailed documentation? It's just too complicate to troubleshoot effectively. I always think admins of any system get better when they know the programming model and detailed information about their platform's implementation, but they shouldn't -have- to know that in order to troubleshoot the simplest of problems.


This is why windows gets a bad name in reliablity; a ton of admins know nothing about how it works behind the wizards and property sheets, so they can't troubleshoot it effectively.


I guess I'm saying that what you gain in productivity from easy out of the box setup, you almost always lose later in frustrating troubleshooting sessions. In unix, this problem would be solved conclusively with a single truss/strace command (or Dtrace if you're lucky enough to be running Solaris 10).


Sadly, I luckily haven't had to deal with IIS > 5 much (or at all), so I can't help Joel out here.

October 28, 2004

The Scobleizer posted regarding choice of mobile phones on Russell Beattie's blog about looking for a new phone. I'm with Tim and Jeremy. I've been chasing the "perfect" mobile phone/web/pda combination since I first tried to actually use the calendar and paid seventy stupid bucks for the "Sprint Wireless Internet Connection Kit" for my Motorola Timeport (which was really just a startac but with a slightly different shape so all the accessories were incompatible).


The main reason that nothing about the wireless web can get me excited is because it's completely carrier controlled. The phone network itself existed for 50 years before any real customer innovation came along, thanks to it's closed nature. The phone company definitely owned it, and hacking was not tolerated. They've done the same thing with the wireless internet. They cripple the phones, removing features that people might actually use. They cripplie their internet gateways to retain control. They require you to have a business plan and negotiate with them to get the details of how to use various parts of the network. Hobbyists and hackers are almost completely locked out.


When I picked up my Sanyo 6400, one of the first things I thought would be cool would be to hack together (for fun) a little webpage that used the 911-location service to stick my currently location on a map on my web page. Nope, not possible. Not only were those features not enabled yet, but in order to do it you had to approach sprint with a business plan to get the details and approval. I'm not even 100% sure that I can download java applications to the phone even though it supports it. Certainly the VM and features are crippled enough that it's never made it worth it for me.


Okay, well lets add some multimedia to the phone, certainly I can just download it over the internet. Nope, that feature is disabled (either on the phone or the internet gateway). You have to buy some software to download ringers or images to your phone, or pay $x.99 a month to get their stupid "ringer" service. My ATT blackberry ships with a crippled internet browser that you can only enable using a strange, unsupported hack.


The hardware is dismal as well. One phone has a decent interface, but doesn't have good internet support. Another phone has great SMS, but has a horrible screen, or some horrible user interface deficiency. They all just manage to suck in some profound way.


The internet exploded because anyone could use iit for whatever they wanted, without restrictions, and they could plug anything they wanted into it. If the network was really open there would be almost limitless possibilities. None of these cooler applications have appeared because it's just too much of a hassle to develop them.


I would -love- a useful mobile revolution. But until there is more open hardware and software available, it just ain't gonna happen. Why don't the carriers just charge for the airtime and get the heck out of the way? The airtight control they keep over what devices are allowed on their network chokes the life out of any sort of real innovation that might happen.


Eventually, things will probably get somewhat useful. But as long as the phone companies ironclad control over the ends of their network, the pace is going to be inredibly slow and frustrating.

October 24, 2004

In the blog Software is too expensive to build cheaply..." (good blog name BTW), the author mentions his favorite weblogic bug, in a posting about Weblogic 8.1 sp3 being out. I thought that would be a fun thread, what's your favorite weblogic bug? I've been using weblogic since 4.5.1, and it's soooo hard to pick a favorite. Is it the one where putting a non-serialized object in the session in a cluster completely brings all session replication to a halt for all sessions? Perhaps our current 6.1 sp5 problem that causes weblogic to throw null pointer exceptions to anyone trying to log in while realm.refresh() is being called after a new user is added. Or the one that causes the application to revert to the initially deployed exploded .WAR, ignoring any updates that have happened since unless you delete all of the staged content between restarts, whch means your JSP's need to get recompiled again...want to precompile your JSP's? Sorry, there's a bug that causes a time roundoff in the precompiler so weblogic always thinks they are out of date and need precompiling... Deadlocks on connection pool obtaining were also fun.


Hopefully weblogic 8 will fare better for us. Weblogic 4.5.1 started to get really stable right around service pack 12-13...6.1 got pretty good after service pack 4...At least I've always liked BEA for putting full release notes, with the issue number for everything fixed, right on their website.

October 23, 2004

So explain to me the logic in spending 15-20 years running around, tearing out useful, simple, centralized, shared mainframe applications on dumb terminals and replacing them with PC's on the desktop, only to try and lock them down so far that the user has no control over the system itself?

October 21, 2004

So weblogic 8.1 provides you with a fancy "split directory structure" which allows you to avoid copying your static content and jsps around to develop. Fairly cool really, but it's managed through two ant tasks, wlcompile and then for deployment wlpackage.



Part of the split directory structure and these tasks is that they assume every subdirectory of where you point the source root is part of the application. So if you want the natural arrangement with most of your source and files right off the root of your checkout, it assumes your database directory, your "build" and "dist" location directories, and anything else you store in there is a J2EE module. Annoying, but in wlcompile you can specify includes and excludes to get around ths. One would think it would be only natural to have the same capability for wlpackage when it's time to deploy your app.


But alas, it's totally brain dead. It sucks up everything from your source root and includes it. So you get your test source code, your database scripts, development tools, even your build.xml all in your EAR that's supposedly going to production. And there's no (documented way at least) to exclude anything from it.



So you spend a pile of money on an expensive app server and you -still- get to muck around with ant's jar/ear tasks to pull the files you need. Thrilling. Even if you follow their example to the letter, you get -everything- included in your production .EAR. They have hardcoded exlcudes for build.xml and .beabuild.txt apparently, but that's it. So unless you decide to be slave to thier organization and put the bulk of your code in a clean subdir off your source root, wlpackage is uselss. Would it really have been that hard to add "includes" and "excludes"?

October 19, 2004

This is pretty damn funny: the CNN.com transcripts of Jon Stweart on Crosstalk. While this may be nothing more than a 2 bit publicity stunt, or someone out of their depth, or not quite as funny as I find it (all of which are doubtful), it doesn't make what Stewart says any less true.

October 19, 2004

A comment on this article on Michael Howard's web log,
IIS6 vs Apache2 Security Defects, got me thinking a bit about the differences between the windows way and the "unix way" (for lack of better terms).

I'm only sort of a zealot one way or the other-use what works for you. But here's an example of where the "unix-mindset/philosophy" differs from windows. If I want to backup an apache install, I can just copy the apache directory. I get all the binaries, all the configuration, and all the certificates. The only thing you'd be missing is the one OS user and group the server runs as, but those are easy to determine from the configuration and create manually. But IIS? What -exactly- is IIS? What makes up the server? Lots of files in system32 mixed in with operating sytem files, maybe some other locations, and they are all mixed in with windows system files. Where is the configuration? Partially in the system registry, a file I can't even make a copy of, and partially in the metabase.

Another example, how would you version control the IIS configuration, so you can rollback changes that cause problems or determine why someone else made a change? In apache, just use whatever version control system you're familiar with to manage the configuration (rcs or cvs come to mind as easy to work with). I'm sure it can be done with IIS, but it probably requires jumping through some hoops and working out some issues. Many admins (rightly so IMHO) consider version controlling configurations to be critical to good system administration.

These are just a tiny example, but in general, Windows seems to prefer complex implementations that are more hidden from the user via some sort of abstraction, and things like Apache are more transparent. Skilled admins tend to like tools that adapt to their workflows, rather than forcing the workflow of tool on the admin. Until that changes, Windows is going to be a hard sell to a lot of people.

It's also been a while since I've worked heavily with IIS 5, so some of this may be outdated, which really I'd love to hear, since I'll pretty much use whatever works best.

October 17, 2004

Tirsen at codehaus says jutopia - Data Transfer Objects makes me sick!. I mostly agree. Objects should have behavior; if you can't define something's behavior in once place, what's the point in OO at all? However, the ony thought I have is about passing the information in a domain object into another "layer" of the app, especially for presentation. Is sticking the whole, full-featured business object into the request a good idea? (on a web app). It doesn't seem like it. I have a few ideas, but I wonder what everyone else does.

October 7, 2004

When are they going to realize that we don't want magic , inflexible, blackbox tools. We want things that it's easy to get diagnostic information out of, and configure to our likings rather than have to bend everything we do to fit the will of the vendor.

October 6, 2004

Scoble asks, what's your product's philosophy?. Currently I'm working on a monsterous behemoth of a web application built by various team members of a big-5 consulting company over several years. When we inherited it, it took 1 hour and 30 minutes to start weblogic, pushes were totally manual and unreliable and took anywhere from 2 hours to 8 hours. Oh, and it crashed at least once a day. It took about 60 days to fix most of that, but we've been living with the almost intractable cruft for quite a while (we're currently in the middle of finally restructuring -everything- in one big swoop).

Our philosophy? "This thing is a soul sucking beast, maybe I should become a fisherman."

Not very inspiring I guess.

October 5, 2004

Okay, they're not really the devil, bad architecture is the devil. With the business layer passing stuff to the presentation layer, which then processes it and passes it back to the business layer for more modifications. Our EJB's depend on our servlet classes, which depend on other EJB's, which depend on some shared classes which depend on other EJB's.


The people ("Consultants") who set this up are long gone. We've made due by following their "strategy" of putting everything in the system classpath, and we're trying to fix that, but it's just about an intractable problem.


Hint; if your EJB's import javax.servlet or servlets of your own, then you probably aren't going to win the "architecture of the year" award.

October 1, 2004

These are autogenerated by eclipse, and I'm always reluctant to check such files in, but it seems like it would be a great aid to setting up all the developer's IDE's the same way. Maybe there's an ant task out there that generates a .classpath or .project file from classpath entries in the ant build file? That would be somewhat cool.

Anyone have a good method of easily supporting both ant and IDE users at the same time, with minimal redundancy in the configuration/build files?

September 28, 2004

It's weird reading through your RSS feeds and seeing yourself mentioned by name. Kinda cool. I think I was originally subscribed after the How to read Smalltalk if you are a Java or C++ programmer. I still haven't grokked smalltalk yet, but that could be because between the upcoming baby, school, a seemingly double-load at work, the backyard, and greedy anticipation over reading the final Dark Tower book, I've only spent about 15 minutes on Smalltalk. I'll get there, I promise. :-)

September 15, 2004

I find the interaction between my mail filter and blog spam interesting. See, I get mailed about all the comments here so I can see the spam and click on the mt-blacklist link and blow away the jerkoffs.

Now, the interaction comes in that usually the post is talking about o n l i n e p o k e r or herbal v1agra. So depending on the post, my email filter might flag it as spam. bogofilter does an awesome job though, so usually I see emails about posts. Every once in a while though one slips through, so usually after a big round of spam I check the most recent comments here as well and remove anything lame.

Spammers. What a bunch of turds. (Can I get that on a bumper sticker?)

September 9, 2004

Hey, check it out, someone agrees with me on the issue of scheduling a formal, nightly build and also having CI builds. Granted, he puts it a lot better than idea, but after much consideration this is the exact system I ended up using.

August 28, 2004

Here's 10 rules I came across for running a project. Interesting reading. I'm not a PM (although I'm a dev lead now on probably 2 projects, and some of the same principles are supposed to apply), but I must say that of the top 2-3 best PM's I've ever worked with, they exemplified and the bad ones missed the mark on most, if not all 10. I'd never really thought about what set them apart, but this article nails it for me.

August 28, 2004

This is a bit more like a press release, than a blog entry from someone at Sun, but I have to agree. The great part about working with Sun, and why I'd keep choosing them for many (but not all) places in any company I ran, is that there's someone who knows. Eventually your support call will end up at that person. I don't end up calling support often, but when I do, Sun is always my favorite to deal with.

August 21, 2004

Ooooh...InfoWorld has the top 6 myths of IT. I think I disagree with everything they say, and they don't even really put forth an argument so there's not even much to respond to. For example, when it comes to "Most IT Projects fail", they take a comprehensive study done of 13,000 projects (just one such study performed in the past 20 years) and attack it with a couple of anecdotal quotes from 3 different people. And still the best that they can come up with is, 50% of projects really fail to deliver significant functionality or features and 15% of those get cancelled or never delivered.

They also have a stunningly obvious graph showing that small projects succeed more often than large projects. Well duh. We've got 40 years of work in software engineering that shows as projects scale up, they get exponentially more complex and likely to fail. Thanks for the newsflash. I mean look at the more than 10 million entry; only 2% of projects succeed. Two percent. And I'd venture to say that projects in that range are the ones most critical to the business. Even at 15% of projects being cancelled outright, can you imagine if 15% of civil engineering projects were cancelled completely? People would be a little pissed if the bay bridge had a 15% chance of not getting finished.

Check out this view of failure:

But AMR’s Shepherd has another view, which he says is more realistic. “Failure would be a situation where orders stopped being taken, or the books couldn’t be closed, or the project itself was simply abandoned,” Shepherd says. “That’s rare."

Uhhh...how about if we define failure as the failure to either a) save more money than the project cost, b) increase revenue more than the project cost, or c) some combnation of the two to get back the project's cost. That should be the real criteria since that's the goal of the business.

Sigh. I hate our industry sometimes.

update: I'm sorry, "X doesn't scale" is a correctly identified myth. The author of that one actually knows something. Neat.

August 19, 2004

Here's a good guide for making long-lasting URL's.

August 5, 2004

Well now I know what the next step in the spam arms race is...no sooner do I install mtblacklist then I get nailed by some fucker posting links to a site not on the blacklist, and the text of the messages is random quotes about programming. At least they are cake to remove now. Still annoying though. Why don't these people just get real jobs, that actually contribute to society instead of sucking off the economy like leeches?

The real sad part to me is how much easier they have it than the people trying to defend against yet, yet spammers still basically suck at what they do. I can dream up tons and tons of ways to make auto-spamming software better at what it does, and rarely do I see them actually implemented. I guess I should be thankful, but it mostly fills me with disdain towards the actual authors of the crap.

August 2, 2004

I finally went ahead and installed MT-Blacklist. Pretty easy, seems to work well. I don't (as of yet) get tons of comment spam, so this will easily handle things. So no more penis-enlargement ads here. Can't wait to see what the next step in the arms race is. Too bad people just can't stop being assholes.

July 29, 2004

On safari, I'm reading Planning Smarter: Creating blueprint quality software specifications". It's pretty good, and eerily talks about almost exactly what we are starting to work towards accomplishing at work. I like the analogy of specifications to blueprints. I think this is exactly covering the terrority that I always thought was missing: "Okay, we've got our requirements, now what's the next step to turning them into software?" I love that he mostly dismisses as metadata most of the things that people hold near and dear to their hearts as the canonical planning documents.

Also interesting was a deconstructing of the notion of software development as science or engineering. There are some good, solid reasons why it does not map directly to either of those, and is much closer to creative writing (something I've been thinking about anyway).

There's a thought in chapter 5 (section: "Hire the Cream") that was interesting to see as well. The author used a piano player analogy, but I thought of it in sports terms. Yes, you can train people and make gains their quality and productivity, but when it comes to finding people at the top of their game, there is a very limited pool available. There are only a small number of people who can play professional baseball or basketball, despite what amounts to a worldwide talent search and training program (school sports). There are also incredible rewards to be had (fame and money) for those with the ability, so everyone has a pretty good incentive to try and make it to the pros, but still only a few can. Basically the talent pool size is fixed pretty closely to the population size, rather than any other training or industry factors, which means it's going to be incredbly difficult to attract and retain any sizable number of heavily talented people, probably forever. Food for thought there.

July 7, 2004

Let me see if I understand the argument posted here: Why PHP Scales - A Cranky, Snarky Answer.

1. Php scales because the programmers are more careful designers, since everyone always tells them that their platform doesn't scale (paragraph 1) and they want to prove everyone wrong.
2. Php scales because it has a "share nothing" architecture.
3. Php scales because the language helps keep you from falling into typical scalability tar pits.

#2 is really the only argument as to why php scales. #1 is a generalization about the programmers which could apply to any platform/lanage, and #3 is a usability issue, not a scalability issue (it's perfectly possible to build essentially the same architecture that PHP uses in Java).

So we're left with the "share nothing" architecture. That's great if you can design your app so it is stateless on the server side. An app that doesn't have to worry about tracking state will always be faster and more scalable. But most full-functioned applications require some notion of state.

It seems that most of the friendster-php flap circles around this "share nothing" idea. But it seems that there is sharing, it's just at the database layer. So it's not really avoiding sharing, it's just pushing the sharing somewhere else. Essentially it's forcing you to build a robust, scalable database layer. Maybe that's okay, because you probably need one anyway. Personally, I'd rather have the design flexibility to scale at other levels as well. Why force the sharing to happen at the RDBMS layer, where you likely incur the overhead of transactions, network, and disk access. Since sessions need some of this capability, there is a certain elegance in reusing the RDBMS for it, but I've never thought performance would be one of them.

Theoretically at least, a model like weblogic's in-memory session replication should be much faster than a full blown RDBMS. Especially when start throwing more and more servers into the mix. 100 servers making reads and writes to a mysql database serving up session info is going to bring things to a crawl rather quickly.

In the end, it doesn't matter much, all we've proven is that you can build non-scalable apps in java and we can build non-scalable apps in php. No news there. The fact that the friendster team seems to be convincied that it's impossible to build a scalable app in java seems to point at some sort of misunderstanding on their part though, that's the only thing rubbing me the wrong way.

Basically, while I don't doubt that php scales, there's no reason Java can't scale the same way, and no one talking about this discussion has produced any accurate arguments as to why java can't scale the same way.

July 2, 2004

Some of the new sun blogs aren't fantastic, but a few from the kernel/solaris guys sure are. Check out this tip about nohup in Solaris 9. That's absolutely awesome; countless times I've kicked off a build (or a java server with inadequate start scripts...just in staging, never in prod) and forgotten to do something with it. nohup -p sweet.

What I'd really like to see in Solaris though, is a way for Java programs to behave more like regular daemons. The whole, background and redirect stderr/stdout or nohup java server processes is hokey. Or am I remembering that Solaris supports direct execution of java binaries somehow? Either way, I'd like to see a setproctitle(3) like call, because "weblogic-application-name" is a lot easier to find in a ps listing than "java -Dblahblah=blahblah -D blah-blahblah -ms128 -mx512 -". Simple process names, easy ways to create daemons (again, I can live even if they require tiny native stubs), possibly reliable jvm signal delivery.

Basically, Java daemons just don't behave like other programs, and usually how they do behave is much more inconvienent than typical unix daemons.

We run a bunch of weblogic instances on the same box, hosting about 10 different applications. That's about 20 java processes (admin servers) on the box, and finding out which one is listening on which port or which pid is which can be a nightmare. It usually ends up as a dance of trying to find the pid of the shell start script, then finding the VM that has that as a parent process. Pretty tough to do from a script. I've been thinking about using 'exec' in the shell so the pid stays the same and I can write it to a file, but then the ps listing is definitely not helpful...

July 1, 2004

So I finally took matters into my own hands. I installed all the dev packages for gtk required to build the gaim source package (one of which removed all of my gnome themes, including the one I was using, gee thanks), dinked with the debian package stuff from the existing one, and made my own gaim. It seems to have worked miracuously, and I'm back on IM. So much for debian as the "Easy-to-configure-and-use-wonder-distro". Thanks to Jan de Groot, who in another language posted the tip.

I could have I supposed just built it from source after I installed the dev packages. I do with installing libgtk2.0-dev hadn't uninstalled a bunch of themes and graphics though; the defaults are prtty flat.

June 30, 2004

I installed debian the other day, and it's been working great for me. I went with the "testing" release because unstable sounds scary (despite many people's assertions that it's fine) and "stable" in ancient with a capital A. It was an easy install (I used the new installer), I got everything I wanted, and apt-get worked great. A few little tweaks were necessary, but it's the first box I've ever got the nvidia drivers working on, and all was good. I thought I was set going forward, since I could always just apt-get my to latest version of things.

Then yahoo changed their protocol, and gaim .77 stopped working. Unstable has .79 now which fixes it, but it hasn't been moved to testing yet, and it's been a week without any instant messenger. If you were on stable, the distro that's supposed to be what you use if you're not a debian developer, you'd be on .58, and would have been without instant messanging for, oh, like a year or so. That's not exactly a rining endorsement for running "stable." Shouldn't critical patches to things that are required to make things work make it to stable?

So debian's got it's share of warts. I'd probably love it on the server, happily running stable, never upgradring, but on the desktop where you need the latest and greatest to even keep up with the rest of the world, I don't know if it's going to work for me now. Pretty frustrating wanting to talk to some friends and not being able to. Windows just keeps looking better and better every day. Should I bump up to unstable? Who knows....

June 28, 2004

My opinon of cc has been dropping steadily since I started using it heavily, first from "cool", to "It's okay", and now today all the way to "sucks". That's not really true I suppose, it's just one team's tool for continuously building, perfectly customized for their types of applications and workflows. But if you're not on that team, then you will likely suffer excrutiating pain trying to get cc to do what you want it to. Today's final straw is that the ExecutePublisher doesn't support any elements to pass command line parameters. So all it does is run a script, with no way to pass any information to that script. Which pretty much rules out using it to do something like deploy the build you just completed. There's also no way to only publish on build success, so the script would have to be smart enough to detect a failed build, which is complicated by not being able to pass any information to the script.


Near as I can tell, you'd have to build a script to get run by a cron job that knew the directory structure and enought to know if the build was sucessfull and schedule it to run periodically. Of course, avoiding crap like that is why I wanted to use CC in the first place. So the cc crowd would say "so just do the deploy from your ant script." Execpt that I don't want the build success/failure to be dependent on the deployment. So no problem, I spawn the task, don't wait for it, and set fail on error to false. But then I'm running out of the checked out directory, and the deployment script kind of depends on not having it's files to deploy removed and rebuilt out from under it.


Where I want to run it from is the copy I make with the artifactpublisher. But there doesn't seem to be a way to that. It doesn't even look like there are parameters to pass to a Publisher to create a patch that will allow passing parameters, without major changes (or ugly hacks). There might be, but it would involve a bunch of JDOM reading of the xml log file, which I'd hate to have to learn for this one simple thing.


And I'm only finding this out after weeks of screwing with cruisecontrol, having to create 4 seperate cc projects for each of my logical projects (1 ci build, 1 master build to do a tag, and then 1 each of those for the branch). Oh, and cc having a major bug in cvs branch modification detections that they haven't bothered to do a release yet to fix.


Anyway, it doesn't really suck, it's just a giant PITA. I had hoped to avoid the pain in the ass-ness of home rolled cron builders and deployers, but at this point I'm thinking about going back to them.

June 24, 2004

Crapola. Firefox 0.9 doesn't manage to keep the icons for sites around correctly. After a reboot of the system, they were gone again. Now I can't think of any good reason why a reboot would trigger this, so I must be missing some other symptom. They also don't seem to work right on Windows. Really, I don't get it; you bookmark a site, so the browser does an http get on favicon.ico. Then it saves it to the hard drive somewhere, indexed by domain name. Then when asked to display the icon for a particular url, it parses off the domain name, looks it up in the map, and pulls the appropriate icon data.

What's so freaking hard about that? Why is it that the bookmark icons continually disappear on every single browser I've ever seen. Is, "displaying customized site icons" somehow a core computer science problem that is just about unsolvable? Is the correct algorithm NP-complete for some reason that I'm missing, or is this just conicidental shitty coding by all the browsers?

I suppose if I'm going to talk shit, maybe I should submit a patch for the stupid problem.

June 16, 2004

Hell has frozen over, I'm convinced. The website icons for Mozilla's released-yesterday Firefox 0.9 browser actually survive a restart correctly. This as far as I know, marks the first time a browser properly handles those little icons, including IE which periodically seems to lose them. I highly recommend Firefox by the way, it's quite fast, has a good feature set, and is simple to use.

June 12, 2004

I agree with the author in that Learning Assembly Language Is Still a Good Idea. However, I disagree that it's a good idea to be writing almost anything in assembly language, unless you're writing device drivers or operating system kernels. We'll start with the benefits. It's important to understand how the machine works; it gives you better insight into the system. It will help you understand how to use libraries and abstractions; what code they generate and the effect of your choices on performance will be. Programmers who understand the abstractions they are using are better programmers, period. The best programmers I think understand them all (all the way down to hardware, at least in some sense; all the way down to the wire when writing network programs, etc.).

However, I'm not clear on when it would ever be a good idea in most circumstances to actually write assembly code in a production project because it's

• error prone,
  
• not remotely cross platform,
  
• maintenance programmers likely won't understand it,
  
• and extremely unproductive compared to a high level language.
  

Obviously if you're writing device drivers or low level operating system code, it's required. But witness how even in the Linux kernel, assembly use in isolated and minimized, and the boot code strives to get into the C code as soon as possible.

The level of optimization that assembly provides is just not appropriate for most software projects. It doesn't really matter how fast your "fooBar" routine is if someone calls it like this: fooBar(new BigGiantObject()) in a loop 10,000 times, where BigGiantObject() has an expensive constructor. Knowing assembly will help you understand why this is a bad idea, which is good, but writing it in assembly will not make it much faster. You want to take advantage of the fabulous abstractions that exist today. Assembly language disallows that. Yes, you can call those libraries, but then you're just writing the glue code in assembly, which isn't going to buy you much performance. Plus maintenance programmers will shake their head at what you've done, and probably throw it out anyway as long as they can.

So knowing assembly language helps you write faster, better code. Just don't write it. Given all things equal between 2 programmers, I'd hire the one that knows assembly, unless he/she wrote assembly code in their interview.

Libraries and high level constructs are not crutches of inadequate programmers, they are abstractions, and abstractions are at the core of what we do, so problems can be simplified enough to solve.

As a final example, one of the comments on the artice (search for OCR) has an example. I highly doubt that the time and effort spent micro-optimizing assembly code was better for the business than just upgrading the PC's. The cost was probably about the same, if not less to upgrade. Not to mention, assembler was probably not necessary. I bet by rewriting the same sections in c++ or c using similar techniques would have provided very close to the speed benefit. I'm sure it was a fun excercise, but it probably was not the most appropriate thing from the point of view of the business. I think dropping into assembly language rarely is.

June 10, 2004

In the last paragraph, This post in a weblog I follow (one from Scott's feeds) got me thinking about something:

The obvious retort is, "Sure, if you can spend millions of years watching failure after failure die, you can make any kind of kludge work, but we have to deliver something next quarter." Which I can't really argue with. But I wish I could. I persist in thinking that maybe there's something interesting about the fact that these fantastically successful systems - bodies - so good at absorbing abuse and coping with change, are not the kind of systems that appeal to us computer people.

Applications often struggle to work in the best of conditions. Apply a little pressure, whether network latency or unreliability, memory or cpu shortage, bad input, bad hardware, heavy load, etc. and they often fall right over. They just aren't very robust. The world at large talks about "survivability" in the context of the military; survivability of an aircraft, tank, or radar system in the face of combat conditions.

Designing and building for survivability accounts for the great expense of procuring military hardware. Designing cheap airplanes is not hard; turn to the kits in the back of popular science magazines to see that. Constructing a plane that can safely fly the pilot home after losing 1/3 of the wing and 75% of the computer systems however, is difficult and expensive.

I think I'm going to co-opt the survivability term (which the dictionary defines simply as the ability of something to survive) and start encouraging its use to describe necessary robustness applications need. If an application can't survive a third party web service hanging, it's not survivable. If it can't survive running out of disk space, having one node of a cluster violently die, or recover from a temporary memory shortage, it's not survivable.

This involves a cost trade-off that needs addressing. Making an application more survivable makes it more expensive to develop and test. However, it saves you the cost of system administrator time later and possible production outages that would have resulted (which could be very expensive, depending on your application). I think very few people make an attempt to quantify that trade-off and make the decision deliberately.

June 3, 2004

I just spent (minus about 7-8 hours for sleep) the past 45 hours focused on one single task; debugging a problem we were having in our testing environment (which was a critical blocker for a fix that needed to go to production). I learned a lot about interperting java core dumps, VM debugging switches, etc.. It's really tricky to debug an occasionally core dumping java app when you can't reproduce the problem. The only data you get is the process core dump, which really lives outside of the java abstraction. Without a way to translate it into a Java frame of reference, there's not much you can do. Even Sun couldn't get much further than I did; they don't have any magic kung fu which will tell you what the state of the java VM was at crash time, without lots of instrumentation and an environment controlled by them. Java 1.5's supposedly got a lot of improvements in thsi area, I'm anxious to see them.

Anyway, with any luck the immediate fire drill is over, and now we get the joy of digging into the codebase to find an intermittent, almost impossible to reproduce infinite loop causing a stack overflow in the Java VM process and a core dump. Oh yeah, and the hotspot error message was nice enough to point out where it was happening...it's in a method called "add()", but it didn't give us the class. That's not all that helpful; too bad it's not occuring in uniqueMethodUsingRecursionWithAnObviousSyncrhonizationProblem().

Update: I just downloaded the 1.5 beta2, and it's service functions are sweet. It's jmx enabled so you can out of the box add a single parameter and monitor all of the nitty gritty classloader, memory, and thread parameters, as well as logging and loaded MBeans. Very nice. Screw the new language features, this is reason enough to be excited.

May 28, 2004

I don't know what the solution is, but it's clear to me we should stop blaming users for picking bad passwords.

Apparently 70% of computer users would trade their password for chocolate, and 34% would for nothing: Would you trade your password for chocolate? | The Register. This article has an "open letter" to users about passwords and is by a horrified security professional.

Personally, I'm not surprised at all that end users don't care about their passwords. There's really no motivation to; I think in most people's minds

• why would anyone want my password,
  
• passwords are really hard to remember,
  
• there are so many of them, and
  
• no one's trying to break in anyway.
  

There's really not many consequences for giving out your password, at least for the person in a corporate environment giving out their password. They've done it tons before and it's never bit them, so they've learned it's okay.

There are a variety of reasons for this, but I grow really tired of the security community's considering users to be "unwashed masses." Isn't it obvious by now that this is a systemic problem and not a "user education" problem. People have bad password discipline because they are not security experts, just like they go 20,000 miles between oil changes. They know it's bad, but still don't understand (or care about) the consequences enough to do anything about it.

May 27, 2004

Hey, check out the improvements they've made to the originally silly bill introduced in reponse to initial reports of what gmail would provide. The first version completely missed the point, but the latest version actually makes sense. The blll was actually improved and made useful in committee; who'da thunk? It explicitly makes it legal to scan email to display advertisements in online services (which was the non-scary part of gmail), while making it illegal to retain that information to build a profile of the user (the decidedly scary, mostly unaddressed implication of gmail).

Funny to see them get something right for a change.

Update: Humm...you know what the bill doesn't seem to cover? You're not allowed to retain any personal information gleaned from the private communication, but it allows to you display contextual ads based on the content. Okay, but it doesn't keep you from retaining the history of ads display to the person does it? Since the ads are contextual, retaining that history still allows you to build a profile...example: Jimmy gets a bunch of email about bass fishing from his friend, so he gets served up bigfish.com and fishradar.com ads while checking his Google mail. Google can't remember that Jimmy got a bunch of bass fishing email, but I don't see that they are explicitly prohibited from remembering that they served him 4 adds for bigfish.com and 10 for fishradar.com. That makes it pretty easy to build a profile that says "Jimmy's a fisher;" really it seems like about the same amount of information leakage as just retaining the context info from the email....

May 26, 2004

This entry is a geek entry, so I'm talking about cruisecontrol the build software, not cruise control the car option. I've been off-track fighting with the thing on and off all day long. It was working perfectly fine until I added some new branch projects, and now it's a nightmare. I get "error reading log.xml" errors all over the place. It's like the log files were overwriting each other or something.

I think I found a possible culprit; there might have been a spare cruise control process hanging around. It's a possibility at least. I killed it, verified that nothing had an open file in the cc directory (it's tough to id those cruise control processes) with a nifty little shell script, and now I'm trying again. If this doesn't work, I think I'm gonna have to call it a night. Which sucks, because this was not what I was supposed to be tackling tonight; I wanted to get the new builds deploying to staging...:-(

May 24, 2004

Software design is a field with scant few public examples of high quality work and analysis and critique of that work. Perhaps that is one reason people have such difficulty with designing good programs?

Great writers learn lots about writing by reading other people's work. artists start by coping the masters to learn their techniques...phographers seem to thrive on viewing the work of others to draw inspiration from. And critics of all three constantly analyze and discuss every aspect of these works.

Where is the software equivalent? No computer science program I'm aware of has a "survey of modern programming literature" class. Certainly there are lots of open source projects now whose design could be dissected and explored for teaching purposes, so as to show programmers both what to do and what not to do. Often the documentation is not in a state that allows easy dissection of the high level design. Proprietary software closely guards their designs, and those designs can only benefit the owning company, unless after the period of commercial usefulness they are returned to the community.

It's not a suprise to me that people have a tough ti