You’re Doing it Wrong If
As a collective industry we’ve been building web apps for about fifteen years now. In many ways we’ve greatly improved our development processes, our tools, our methodologies and our expectations. Today we whip out Rails apps in days that would have taken months before; single developers are producing entire commercial sites where once a team would’ve been required.
But alas not all is golden in this day and age. Time and time again I come across development practices and code that would not have been out of place in 1995, and that is not a compliment (and incidentally, these practices and code would have been bad then too, we web developers were just too divorced from the comp-sci folks to know it at the time).
Fortunately experience and hindsight can now provide us with rough guidelines against which we might develop our own apps. In the spirit of fifteen years of web development advancement I present to you “You’re Doing it Wrong If”.
You’re doing it wrong if:
- You’re not building atop an open-source framework
- You’re writing your own framework upon which to build
- You’re not using a database abstraction layer
- You’re writing your own database abstraction layer
- You believe you can write it faster or better or more efficiently than any available library
- You have more than one developer on your project and no written coding style guidelines
- Your bug tracker serves as your functional scope or your development roadmap
- You aren’t using source control
- You comment the what but not the why
- You tend to pass properties instead of instances to functions
- Your “deployment procedure” involves any combination of FTP and/or drag n’ drop
- You write code in a manner that cannot be unit tested
- Your primary method of code reuse is copy/paste
- You don’t read any development blogs focused on your primary and secondary languages or technologies
- You haven’t been to a conference or birds-of-a-feather meeting in the past year
- The only code you read at work is your own
- You worry that some day someone else will read your code and know it was written by you
and finally
- You’re doing it wrong if you aren’t learning every day
Only truly terrible places are victims of all of the above, however no one is immune. The more hurried we are, the more stressed we are, the more of a monoculture we work in, the more likely we are to become affected by these ailments.
(It goes without saying that these are general rules of thumb, applying ye olde “90/10″ rule and you, of course, are part of the 10. However for the sake of everyone else I urge you to consider Bruce Schneier’s famous words: “anyone who creates his own security protocol is either a genius or a fool”. The odds are not in your favour).
Update: Changed “ORM” to “database abstraction layer”.
rogeraaut responded on 28 Jul 2008 at 11:30 pm #
nice list!
i’d speculate, that you are doing it wrong if you still believe in code reuse in the sense of replication of source text instead of model reuse and patterns.
my favourite litmus test is this: how do you think about richard garbriels ‘worse is better’?
chris responded on 29 Jul 2008 at 12:46 am #
Roger: I like that “code reuse” observation.
As for worse is better, if you’re referring to “…in which simplicity of both interface and implementation is more important than any other system attribute…” (cribbed from Wikipedia), I am a big fan of the simplest solution with the least amount of the simplest code possible.
Unfortunately that is usually the hardest solution to implement, all the more so for web developers violating any of the observations above.
Will responded on 29 Jul 2008 at 7:23 am #
If everyone listened to you, there would be no innovation in libraries, ORMs, frameworks, etc. Where do you think Django, Rails, etc came from? It really pains me to see this sentiment repeated throughout the blogosphere.
chris responded on 29 Jul 2008 at 8:20 am #
What I’m suggesting here are rules of thumb for building web apps and not absolute edicts handed down from on high. If you want to actually set out to build a library or a framework… well remove the one suggestion that deals with that aspect and follow all the rest and you should be just fine.
someone responded on 29 Jul 2008 at 8:51 am #
I think building your own ORM, web framework, and library to beat other libraries is a fantastic educational experience, if done in your spare time and merely to satisfy your own curiosity. When you go back to using someone else’s work you’ll look at it differently, because you really understand in a way that can only come from constructing it yourself.
Using that for client work or doing it on someone else’s time and money, however, qualifies as doing it wrong.
chris responded on 29 Jul 2008 at 8:59 am #
We are in agreement on the value of building tools, no question. And if your goal is to build a new hammer, then the best way to go about it is probably to build a new hammer.
However if your goal is to build a house then forging your own hammers when Home Depot is just down the street is not likely the best use of your time.
anonymous responded on 29 Jul 2008 at 11:06 am #
You’re not building atop an open-source framework
You’re writing your own framework upon which to build
Blatant, wrong, and I stopped reading at that point. What the hell are you thinking?
The best practice is to actually build yourself a small-scale framework. You’ll always use a framework if you use a modern language. There’s no getting by it. Python comes with a standard library and specifications that can serve as your framework, or you can climb up the abstraction ladder and end up with Django or Pylons or whatever.
Bad. Post. Die.
someone responded on 29 Jul 2008 at 5:21 pm #
Why is it bad not to use an open source stack? Well, see, e.g., http://www.tbray.org/ongoing/When/200x/2003/07/12/WebsThePlace - you don’t want to be a sharecropper on Massa Bill/Steve’s farmstead, who can unsupport/NDA you out of a business and livelihood at will.
Many applications deserve a domain specific language but don’t have one. Many applications could gain in maintainability by being refactored; for instance; you could refactor out auth&auth logic to be its own plugin, say. However, I don’t see how that amounts to “it’s best, in general, to build your own small framework”. And it isn’t something you’d do in advance.
Shey’s Rebellion » Blog Archive » You’re doing it wrong. responded on 29 Jul 2008 at 11:31 pm #
[...] Sometimes, we need someon to remind us, we’re doing it wrong. [...]
You’re Doing It Wrong If : An Experiment in Scotch responded on 30 Jul 2008 at 4:47 am #
[...] many ways are you doing it wrong? I think I count a definite five and possibly up to seven ways. Does Linq to SQL count as an ORM? [...]
Will responded on 30 Jul 2008 at 5:25 am #
I would actually argue that setting out to build a framework is worse than building one while completing a web app. If you actually look at the examples I cited (Rails and Django), both of those were extrapolations from real world projects. You are going to come up with a much more focused and usable solution by making it do exactly what needs to be done for a few projects than to try and hypothesize every possible option and make it generic and watered down.
The real kicker is “You believe you can write it faster or better or more efficiently than any available library.” This screams mediocrity. Honestly, out of all of the open source libraries and frameworks I have seen, especially web ones, I have always seen opportunities to do it better or more efficiently. Far too many APIs are cumbersome, confusing and inconsistent. Documentation is usually terrible to non-existent, or things are done in such a way that it can’t be easily extended or modified to fit your needs.
Paul W. Homer responded on 30 Jul 2008 at 6:54 am #
Mostly I like your list, but I disagree with your assumption that if it exists already, it is better, or faster for you to use it. That’s a key strategic choice, that just isn’t that simple. Writing it yourself gives you IP, control and eliminates a dependency, all three can be critical to your success.
Deeper IP is worth far more for a commercial product, both for short-term business dealings, and long-term direction. If you’ve written the code yourself, then fixing it only takes a few minutes, if not it could be a lot of work. Often, with free libraries, just after you start depending on them, they get abandoned and start to rust, causing huge resourcing problems. In each of these cases, a simple dependency becomes a serious liability. If you’re spending all of your time trying to figure other peoples bugs, you don’t have any left over for the real work.
I don’t think developers should idly jump into writing all of their own stuff, its way too much work and time, but sometimes on some occasions it is a far better choice for the project if you just spend a few months building something, instead of relying on someone else to get it done for you.
Also, and this is a big also, sometimes depending on knowledge and experience, a custom approach may be way way simpler and more effective than the general solution. Why suffer a multi-million line dependency, if you got the skills to pound out 50,000 lines that works better. Just because everybody has taken a particular tack with their designs doesn’t mean that it is correct, or that there isn’t some better way to handle it. Too many MVC2 frameworks take more than they give, for example.
Paul.
http://theprogrammersparadox.blogspot.com
Pål responded on 30 Jul 2008 at 7:57 am #
I disagree with the two points on ORM, but otherwise a nice list. I disagree on using ORM at all, read the Vietnam of Computer Science.
IllegalCharacter responded on 30 Jul 2008 at 8:33 am #
> You’re not building atop an open-source framework
> You’re writing your own framework upon which to build
Please, tell me a web framework that can handle several million hits per day, supports memcached, and database clusters. It has to be PHP too, since the company can’t afford to switch languages.
I tried Code Igniter, and it didn’t offer much productivity boost over the helper code that we had written already.
I would love to hear about this.
chris responded on 30 Jul 2008 at 8:45 am #
@IllegalCluster: we can all come up with edge-case and one-off scenarios to invalidate each suggestion, but that’s not particularly helpful or practical. I suggest to you that the number of projects taking longer than they should to implement and costing more than they should because developers are attempting to unnecessary write existing components and code themselves vastly outnumbers the number of million hits/day sites that need custom code-level tuning from the very start.
@Paul Homer: “I disagree with your assumption that if it exists already, it is better, or faster for you to use it”
I don’t assume that someone else’s is better or faster, the only way to know is to benchmark and then implement. My assumption is that speed of development and time to market is more important for the vast majority of web apps than library-level code optimizations. And that if a library is in general use and available then it is probably good enough to get you implementing. Someone else has already incurred the up-front time, effort and cost debt so why not take advantage of that? And if you’re wise enough to use open-source products you have the ability to benchmark, modify and improve at will.
Of course there are areas in which something you need won’t be available, we all run into this all the time, and at that point you’ll need to write your own. But unless you’re building something truly, fantastically revolutionary those moments will be the exception in any given web project and not the rule.
@Will: “The real kicker is “You believe you can write it faster or better or more efficiently than any available library.” This screams mediocrity.”
Interesting, since some of the best code I’ve ever seen and which has taught me the most has been inside libraries I’ve used. It takes a certain extreme hubris to automatically assume your code is innately better than everyone else’s, no?
Stephen Johns responded on 30 Jul 2008 at 9:18 am #
I think that you’re wrong about using an ORM layer. The correct way is to use the proper tool for the job. In this case, use an object database.
Paul W. Homer responded on 30 Jul 2008 at 9:33 am #
“Interesting, since some of the best code I’ve ever seen and which has taught me the most has been inside libraries I’ve used. It takes a certain extreme hubris to automatically assume your code is innately better than everyone else’s, no?”
The tongue in cheek answer is: Not really, if I’m a better than average programmer, my code is better than half of the stuff out there. If I’m in the top 10%, then so is my code
More seriously, I’ve been through lots and lots of code, and most of it is in need of some type of cleanup or refactoring. It’s messy, and often poorly structured. And it usually gets worse as it gets older and more popular. A small experienced highly disciplined team should, on average, be able to write better code, it just might not be worth the effort.
Paul.
chris responded on 30 Jul 2008 at 9:58 am #
Ok, perhaps I should have added a caveat to the “don’t write it yourself if you don’t have to” point: you’re doing it wrong if you blindly use other people’s code without understanding what you’re getting.
When it comes to third-party libraries a pretty good rule of thumb is: if development projects I respect are using it, and I respect the developer who created it then I should probably check it out before duplicating it. There’s no short-cut for doing the leg-work. Code is no different than anything else in life, only buy it from reputable sources (and by “buy” I mean obtain or acquire, not a literal ‘purchase’, for the pedants and literalists in the crowd).
Smokinn responded on 30 Jul 2008 at 10:33 am #
@chris I happen to know IllegalCharacter (I imagine we both got here from raganwald’s rss feed links) and know that he does work on a website that gets millions of unique hits (I would be surprised if it was billions of raw hits total) a day so it’s not a one-off or edge-case case for him it’s his use-case, a very basic requirement.
I think the reason people are complaining is because of the target audience. I imagine most people glossed over that you say “we” it’s in opposition to “comp-sci folks”. Comp-sci folks can and routinely do write better libraries than are available as open-source. The company I work at has a proprietary xslt transformer and cacher that’s blazing fast. All written in C. When I was first interviewing and they said they used xslt my first reaction was isn’t that slow? Here I was shown that xslt is only slow because the available open source tools suck. And now that it has a reputation for being slow no one uses it.
I wouldn’t trust a front-end web developer who self-taught him/herself html, css, a bit of php and photoshop to write the kind of high throughput/low latency library a major web site needs and I imagine that’s the kind of audience you were targeting. The kind of audience that works as a freelance web developer for example. They may very well be excellent web developers and would make a much better looking website than I ever could but they’d be better off using existing frameworks and orm software rather than building their own.
While the above holds, I object on your points because most existing frameworks just can’t handle the kind of high load website I work on. I do nothing but back-end work and need a framework to be fast. That means stripped down to nothing but what’s essential and with support for things like distributed processing (for asynchronous processing of message queues for example), memcached and database clustering. There is no framework like that available that I know of.
Other than your first 5 points, good article though =)
chris responded on 30 Jul 2008 at 10:52 am #
Perhaps then I should have added yet another to the end of the list:
You’re doing it wrong if you take these as gospel truths without understanding the specifics of your needs.
cease responded on 30 Jul 2008 at 11:00 am #
“You tend to pass properties instead of instances to functions” -
By doing this you make your code more fine grained… meaning that it makes the code even less reusable. Depending on what you are building.. I prefer to work the other way around. Make it coarse grain, then move towards instances as needed.
chris responded on 30 Jul 2008 at 11:17 am #
It does amuse me to no end that the biggest objection people have centers on the points that suggest developers do less tedious, duplicative work. Amusing because implicit in that suggestion is that by doing so they’d be in a position to do more of the unique, interesting work on their projects.
IllegalCharacter responded on 30 Jul 2008 at 6:04 pm #
I actually think writing frameworks and ORM stuff is fun. It is the code that they are designed to replace that is tedious and duplicative (ie. writing code that generates gigantic INSERT queries is more fun than writing the INSERT queries themselves).
Sorry if I sounded sarcastic or like a flaming troll in my previous comment, I was actually asking if you knew of any high-performance or highly scalable web frameworks since something like that would make my life a lot easier.
Kia Kroas responded on 30 Jul 2008 at 8:11 pm #
I’m just a starting web programmer of a few years and at this point, I find it quicker (and more secure) to code my own framework and abstraction layer. The time it takes for me to set up a website from scratch would be less than the time it takes for me to read up on some frameworks and abstraction layers. Factor in the security considerations you have to keep in mind and times when you need to patch/update the code and you’ll see why it’s worth it to code your own IMO.
The thing is though, my work isn’t that advanced, so I don’t need the features in the big frameworks. Coding your own framework lets you utilize only the features you need and maximize the algorithms for that specific use. (Which I think some of the previous comments have already mentioned.) When the time it takes for me to learn the framework would be considerably longer than it takes for me to code my own, THEN I’ll start using them.
Code reuse is good, but reuse of bad code (or not knowing how/why the code works) is bad practice.
chris responded on 30 Jul 2008 at 8:46 pm #
@IllegalCharacter: Unfortunately I don’t for PHP. Maybe someone else reading here does?
Phil responded on 31 Jul 2008 at 3:55 am #
“You’re not building atop an open-source framework”
Should be replaced with: “You’re not using the best tool for the job.”
Who cares if it is open source or not, as long as it makes my life easier? Open source is fine and dandy, and more often than not a good alternative, but it is not a benchmark (if it is, then for bad documentation and developer / geek hubris).
AC responded on 31 Jul 2008 at 5:22 am #
Interesting how some of these go directly against Joel’s recommendations. He wrote a really long post about how Excel avoided external dependencies and was better off for it. Not to mention his shop has their own bizarre web language they wrote.
Rob responded on 31 Jul 2008 at 5:54 am #
Rules are great. There’s no way that any developer could remember exactly what to do in every situation. Instead, you remember a general rule (to cover most cases) and you remember when and why you should make exceptions to the rule (to cover the other cases).
But the rules are the easy parts. We know them. There’s very little speculation about them. The problem is figuring out when and why an exception should be made. If all you give us is a list of rules, it looks like you’re the one who swears by this list.
Anyone could have written that list. Frameworks do some of the work for you? No kidding! Give us an example of how a particular framework helped you, or give us a situation where you decided it was better to write something yourself. Just give us some insight about something, otherwise it’s just another crummy blog post that has 500 words but not a single thing worth reading.
chris responded on 31 Jul 2008 at 9:43 am #
@AC: “Interesting how some of these go directly against Joel’s recommendations.”
Bear in mind that I wrote “web app” for a reason. FogBugz is more like conventional software in that it is designed to be deployed across multiple operating environments. It just happens to be web-based. Hence the use of their own language to generate language-specific code (PHP/VB etc.).
In such exceptional cases it makes good sense to be exceptional. If I made be so bold as to put words in his mouth, I doubt Joel would suggest everyone take the same approach.
Excel too is compiled software and they only avoided external dependencies if you ignore the vast framework of pre-existing libraries and code that MS had at the time. Just like OS X developers ignore external dependencies only if we choose to ignore the entire Cocoa framework.
I wonder: I bet it’d be pretty rare to hear Windows developers or Mac developers arguing that you should ignore .Net or Cocoa and write your own frameworks, using the arguments the nay-sayers have made above (”those Apple engineers are stupid. I can write my own text-rendering code way better and faster than Apple can!”)
Diego Carrion responded on 01 Aug 2008 at 6:32 am #
You’re Doing it Wrong If
You dont write unit/functional/you_call_it tests
Paul Keeble responded on 01 Aug 2008 at 10:18 am #
Alas your preaching to the choir. The very audience that would benefit from rules of thumb such as these doesn’t read blogs. The very few that do very quickly pick up and surpass these sorts of rules.
I think we underplay how clever as a group developers are. There are some terrible ones sure but for the most part someone in the team can see a better easier way to do it, the problem is always time and priorities.
Joseph A. Nagy, Jr. responded on 06 Aug 2008 at 6:20 pm #
I hate to admit it, but I’m guilty of more then one of those ‘crimes’. I’m working on changing, though.
Ole Phat Stu responded on 07 Aug 2008 at 5:33 am #
Apart from in the dictionary :
Usability comes before Reusability!