A Web Developer's Personal Projects

It's interesting to note that many web developers go through a similar process over the course of their careers. One of those rights of passage is the building of a particular application type. Cutting your teeth on building a larger scale application (at least, larger than a contact form that emails somebody and larger than hacking up an install of WordPress) seems to inevitably involve developing a custom application, either as a personal project or as some client deliverable.

Most importantly, I believe that a tried and true web developer should go through this process to understand the pain points of developing larger applications. There are three different types of applications that are certainly great excercises:

  1. The forum
  2. The blogging engine
  3. The content management system

Each of these often has a number of concerns that are always pertinent — such as user experience design and workflow, authentication, spam handling, email management and certainly database design.

The Forum

Over the years, I've built a few forums. Forums can be simple or they can be complex. Software like vBulletin or phpBB have grown to include a massive number of features, most of which are overkill for the average forum. There are certainly some critical design decisions that have to be made such as threaded or inline thread replies, thread notification, thread stickies, and user registration.

The Blogging Engine

Forums are probably the least popular application type these days as most people seem to prefer building a blog engine instead—the similarities of which are interesting. Both have a main topic with the ability for people to respond. The main differences are in who's allowed to post (everybody vs admin) and who's allowed to reply (registered vs anonymous).

Over the last few years, the blogging engine has certainly become one of those things a web developer just does—usually as a result of outgrowing whatever engine they were using before.

Blogging engines can be much more complex than a forum, though, including trackbacks and RSS generation. (Not to say that RSS couldn't be in a forum; in fact, it probably should. It's just more prevalent in blogging.) Many web application frameworks like Ruby on Rails, CodeIgniter, and CakePHP include a tutorial on how to build a blog.

The Content Management System

The pinnacle project of a web application developer is the custom content management system. Born from the frustrations of trying to wield open source or commercial applications, it seems to be every developer's duty to try and build a custom CMS. How do you manage different object types? Templating, authentication, workflow, internationalization and localization are just a few of the issues you have to contend with in building a CMS. Do you go simple or do you go complex? How does the average user, with little web experience, find their way around the application?

Building a custom CMS is a fantastic excercise on so many levels — well beyond what you run into with a blog or forum. Much of my design intuitiion came from this type of development. Custom CMS development almost always comes from design issues and not technological issues. They are a response to poor and confusing interfaces. Above and beyond forum design and blog design, CMS design really requires you to step outside of yourself and understand what the pain points are and how to solve them. This might also explain why most of them seem to suck and why developers around the globe continue to feel the need to reinvent this bumpy wheel.

But it's also the avenue with seemingly endless possibilities. Is it hosted, is it custom, is it for the designer, is it for enterprises, and so on and so forth.

Any others?

What type of project do you feel is a great way to grow as a developer? As always, comments are open.

Published September 09, 2008
Categorized as Opinion
Short URL: http://snook.ca/s/910

Conversation

56 Comments · RSS feed
James Paden said on September 09, 2008

Excellent post Jonathan. I would add that two other application types that I've developing in the past. The first is a simple shopping cart and the second is an ad management system or classified ads system.

I think both of these address issues dealing with larger applications as well as issues such as payment management and order status.

Usability is extremely to key to keep in mind when working on these kinds of projects. A developer should never lose focus of the end user ("don't make me think").

Chris Fullman said on September 09, 2008

Great post Jonathan! James has a good point... e-commerce and ad management systems both play a part in the bigger usability/site integration role that any developer should know beyond forums, blogs and CMS's.

One big exercise, however, is developing an API; Data exchange is proving very important on how the industry evolves. Knowing the ins and outs of authentication, permissions, rate limiting and information access will help anyone grasp a good understanding of how applications work and behave.

Tony said on September 09, 2008

The forum/blog/cms seems very similar in the fundamental purpose (they are all ways to manage text). How about a voting system, or a photography system, a social network, or an RSS reader? And I definitely second the e-commerce system mentioned above. (Your comment system is awesome btw! I like the realtime view :-)

Todd Rafferty said on September 09, 2008

Simple shopping cart I can agree with. I disagree with ad management systems or classified ad system. It's nothing more than data in, data out.

IMHO, I think some sort of application that relies on scheduling or a release date would be good to have, but that can be mingled in with the CMS system if you really wanted to.

Or, if you really wanted to be cruel, event calendar with the ability to do recurring events. ;) Just make sure you have a project manager that quotes, "Outlook can do it, why can't you figure this out?"

Grant Palin said on September 09, 2008

Several years ago I took a try at building a - really - basic blog system. Then I discovered WordPress. For the most part that has worked well since then. But I have been meaning to go back to that project and finish what I started. And go further than planned before.

I've been toying around with some CMS ideas as well, regarding structure and modularity. Haven't gotten any further than that...maybe some day.

I agree that those can be good pet projects to have, to goad us into learning new tools and techniques, and to refine our work.

Radoslav Stankov said on September 09, 2008

The CMS, is I think, is the must build for every web developer. And if it have modules in it self like forum, blog, shopping cart is really great.

I have make something different, years ago I create a little turn based web strategy for me and some friend of mine, just for fun :) in my school years. Some of the things I learn there how to do, and most importantly not to do, help me a lot over the years :)

Luke Visinoni said on September 09, 2008

I hear that Todd (about the calendar system). I had a go at writing a calendar/scheduling application with recurring events. I have yet to finish it.

Mike Smith said on September 09, 2008

I need & want to start writing custom wordpress plug-ins that turn wordpress into a classifieds website or something else really big, as well as ad management, ect. But I haven't sat down and learned enough about writing php manually yet. I know the basics and I know how to hack php code pretty well, but for some reason, I can't write it from scratch yet. Time will tell....

Dave Rupert said on September 09, 2008

wow. you nailed me to a "t". i felt like i was reading a blog post about my life!

Chris Huff said on September 09, 2008

I'm kind of working my way backwards through this list. I've been working on my custom cms for years now. It's evolved from very simple, to something that is becoming fully-featured, yet with a (hopefully) very intuitive interface. I'm just now including blogging capabilities into my cms as well. A forum may come eventually, but not for the foreseeable future.

Sometimes I do feel like I'm merely re-inventing the wheel, but like you said, it's a bumpy wheel (sometimes it feels more like a hexagon). I constantly think about giving up on my project and focusing instead on giving wordpress a simplified interface, but then I remember it's not just about the interface.

Anyway, you asked about other projects that a web developer grows through. I know for me, I grew a lot when I created a simple shoutbox. It became the basis for my first guestbook, which became the basis for the first version of my cms. It's amazing how all of these things are related.

Anders Ytterström said on September 09, 2008

haha, funny. I built stuff in the exact order (first forum, then blog, now in an ongoing cms development)!

Before the forum I built a guestbook, with admin, form and listing. That's the main reason why people started to learn server-side 8-10 years ago.

Ritesh said on September 09, 2008

I completely agree that building a CMS is one of the best way to learn web development. Not only there are so many things to take care of, but it also gives you an idea of how to organize database tables so as to minimize database queries and how to effectively use caching.

dedenf said on September 09, 2008

aha! i've build two of your list, blogging engine and CMS, all of them using CodeIgniter,

Tanny O'Haley said on September 09, 2008

How about web applications for business needs? Enter data, pull data from other sources, validate complicated forms. Typical stuff a developer has to do at a major corporation.

John said on September 09, 2008

An ecommerce site is good for web developers because it requires you to address issues like security, encryption, and web services for credit card processing.

Eddie Welker said on September 09, 2008

If you want to grow as a developer, try writing a basic web framework. Seriously. You'll learn a lot about how core technologies actually work, why most frameworks behave/are structured the way that they do, and it will certainly straighten you out if you are used to writing even slightly sloppy code.

Tarique Sani said on September 09, 2008

A multiuser photo gallery worked great for me currently it lives as Coppermine Picture Gallery - but a word of caution - many times the immature design decision which are made in the beginning continue to haunt you forever and somehow never get fixed!

Daniel Craig Jallits said on September 09, 2008

I would suggest "Rolling One's Own Lifestream", as it involves working with a chosen scripting language and a multitude of APIs and data formats, and I am not talking about using pre-built code like SimplePie either.

There is no need to drastically change the overall functionality that users expect from your site, as some did when site owners went from Forums to Blogs, as LifeStreams can be easily integrated into pre-existing sites.

Matthew Geddert said on September 09, 2008

I believe that the best project for a web developer is one the developer really wants to do, and wants to do well. Something where you say "wouldn't it be cool if..." that you haven't seen done before. This will force you to figure out how to solve a unique problem that hasn't been documented in thousands of places. It certainly helps to know how to roll your own authentication/email/spam system, but, the best developers are the ones that aren't constrained by those conventions.

Adriaan Nel said on September 10, 2008

I've built an invoicing system tailored around a small business' needs. I use it myself and have a lot of clients that also use it. Before I started, I checked current systems on the web, but none had a couple of features I required, like automatic generation of recurring invoices - especially useful if you do hosting...

Joe Siao said on September 10, 2008

I am making a questions and answers site in cakephp 1.2 RC2 and it is my personal project. I initially thought of creating a blog in cakephp but I felt it was too cliche.

Sasa Bogdanovic said on September 10, 2008

One other functionality which I had to develop with almost every web application I have been working on is a reporting system.
Sometimes they are simple, sometimes they get a bit more complex, but they are always needed if there is any kind of data collection in the application. The most important is to understand which are the relevant data the client need, and to find a way to present the data in readable way for the user.

Navigation in reporting? It can be even more complex than website navigation. Besides a set of navigation menus, I also always use hyperlinks in the reports as a very efficient way to dig deeper through the data.

Ivan said on September 10, 2008

I think you summed it up pretty good. I've went through the same process when starting with PHP, and doing it again with Cake.

Damn, I hate feeling like a newbie :)

dr. Hannibal Lecter said on September 10, 2008

I think the future lies in specialized CMS development. Not just because I'm creating a specialized CMS at the moment:) but because CMSes tend to end up bloated, just like forums and blogs.

Funny thing though, even though I'm develping a CMS, people often congratulate me on great "blogging software".. :-/

What about a combination of all three of those as an exercise? Or the ever popular social networks and bookmarks site?

Andy Kant said on September 10, 2008

I would definitely agree that developing a CMS is a great exercise, as I tend to prefer developing a mostly new CMS (re-using some components such as user authentication systems) for pretty much every website I make. I prefer to develop a basic CMS that is scoped for a specific site and can be bent any way I choose rather than use a CMS system has too many features that I will never use and is difficult to extend the way I would like.

Usually when I'm testing a new technology (such as Ruby on Rails or Django), I do something a bit more ambitious so that I can learn more than just the technology. An example of this was an Ajax-based multiplayer gaming system I developed to learn Rails (turns out that Ruby makes game logic quite a bit easier).

Binny V A said on September 10, 2008

Usually CMS and Blogging Engine goes into one category.

Some other exercise projects...
Digg Clone
Social Networking App
.
.
A Framework

By the way, I have not created a forum yet.

Jack Keller said on September 10, 2008

CMS, sheesh, how many times have I written and rewritten mine, then forked it off for client projects. I would say it is definitely an essential project for any developer to put in their arsenal. Another fun project could be along the lines of a Framework as stated previously.

Ain Tohvri said on September 10, 2008

Surely a great post. I couldn't agree more to the CMS part of it. The work on a custom Content Management System is really endless.

Michael McCorry said on September 10, 2008

I've got my own home-grown CMS that I've been using and upgrading since 2000, plus a forum, e-newsletter management system, photo library flickr-style thing plus a virtual tour management system, but the one thing I seem to use the most is a generic catalog cms thing, which gets used to manage non-webpage content such as real estate listings, used cars, retail products, announcements and classified ads. Now all I need to do is tie all of these things together into my main CMS via some kind of plugin architecture.

Miles Johnson said on September 11, 2008

I agree whole heartedly. Ive built my own forum, user system, blog system, project manager, ecommerce shopping cart, photo gallery, cms and am currently building an online social community. Its the only real way I know to learn more and more, it helps.

Up next I intend to build my own lightweight framework, not a fan of a lot of the MVC frameworks.

Too bad im too scared to apply for a PHP job.

Kim Joar Bekkelund said on September 11, 2008

I have been through these myself. Lately I have started programming a lot more in Python and Ruby, but haven't found a pet project yet. I thought it was a little bit to cliche to actually implement a new CMS, forum or blogging engine. I think I'm in a creative down period. Hopefully it will come back to me soon.

Nikola said on September 12, 2008

Jonathan, I couldn't agree more, those three applications are a must in one's experience. I would only add that extending available applications (CMS, Forums, etc.) with modules is also a must. It teaches how to use API and it teaches of problem solving with limited resources.

Brad Harris said on September 12, 2008

Along with a few others here, while building these applications, I think its a great exercise to extract a framework from the way you develop your applications, if you're not building on top of a framework already. After time you will either end up with a nice custom little framework of your own, or realize you're building CakePHP, or some other framework, and give in to using whats out there. Either way its a win situation in my book, you really learn a lot as you try and create a framework.

Micheil said on September 14, 2008

hmm, I must say I learn things slightly differently. I've got about 30 projects currently open personally, the biggest or best learning ones I've got for PHP are:

1) Photo Gallery – something simple, file based, nothing complex.
2) Something to Manipulate graphics using Imagemagik
3) Server Administration console (similar to PHPMyAdmin)
4) Fully fledged Templating system / MVC / WEb Application framework.

I find these are the projects that helped me learn things the best, so far out of all those projects, I've gotten knowledge on: Mysql, Imagemagik, Classes, OOP, Wireframe Design, and a bunch more!

Brade said on September 14, 2008

So true, man. I've learned so dang much by working on these types of software projects. I've already got two CMSes under my belt -- one that manipulated flat files, and one that is database-backed. For the latter, I had to build in basic blog functionality in only a week's time. Stressful, but I think some measure of success was achieved. Feel free to check it out and see what we've done. It's got some drag-n-drop ajax niceties in an attempt to make things intuitive for most users. Sadly, the company itself is struggling (only 2 guys, after all), but the things I learned give me ample confidence as I forge ahead in this industry.

Alexander Berglund said on September 17, 2008

This is very interesting as I recently contemplated the same ideas and thoughts...

Many developers - as they start out - seem to be very interested in building their own small community. This also giving them an exercise in managing flags, database design, user registration and CMS (on a small scale). These projects only live on for so so long until the developer finds that there actually takes a lot of energy to keep it going and soon abandon the project or build it together with a personal page/portfolio.

As also been mentioned in the comments, a voting system and some form of statistics system also seem to be a popular personal project...

Brade; I haven't dug very deep in it, but I find the content (Text primary) to be slightly amateurish although the rest tells a different tale. It gives a kind of mixed feeling even if the system and graphics looks good...

THEODIN said on September 19, 2008

I Myself have been through this right of passage, starting off small is great, but as time goes by more challenging opportunities almost becomes a birthright, I feel that modularity is the key here, breaking down the most complicated of applications into the componants that run them.

Shaal said on September 19, 2008

A developer should come up with not only apps, but softwares, portals and bla bla bla!!! You're already a grown up developer. Try games for now;)

matt said on September 23, 2008

E-commerce storefront/admin.

John F Croston III said on September 24, 2008

The first application I first built was an RSS feed reader in Classic ASP with the ability to do key word searches to help narrow the feeds from places like CNN, BBC, Washington Post, etc.

After that I started building a bunch of other projects over the last year or so, that had been already done by others like bookmarking, contact lists, wine inventory, and a few others again in Classic ASP.

I'm now going to take sometime off work (one week) instead of going to "An Event Apart" in Chicago and build a few of these applications in PHP and MySQL which I don't currently know. This way I have a great deal of time to spend building them and learning the new skills to help improve my resume. I have a lot of the DB design input FORMs already built, just need to do the conversion to the new language and DB. This should make the process a bit easier.

The other big issue I want to do with these applications is make them accessibile and useful on phone web browser.

I have been fortunate enough to have two different friends that said I can use some of their studio space. I know if I stay home to do the work I will end up watching TV, napping, go to the movies, etc.

I have been reading up on both PHP and MySQL and it should be a fun learning experience if nothing else.

Wish me good luck.

Acorn said on October 08, 2008

I used to work for a company as a Web developer, but I quit my job. Now I only make sites for myself. Its fun developing personal projects than for the company.

Barry said on October 11, 2008

I've also gone through this rite of passage : Started off with a CMS, moved on to a Forum, e-commerce, custom framework... the usual. Been doing so for close on 10 years now.

So far though, one of the biggest challenges I've faced is writing a subscription based service. It sounds easy enough at first, but can get rather complicated. Keeping track of multiple paid for subscriptions per user, all while making sure the system is easy to use proved quite daunting.

bazet said on October 15, 2008

My first project is actually, Web SMS management system using PHP/PgSQL using PHP 3. Since then I've built countless of CMS,customised Equestrian scoring system. Now with cakephp, I write a simple forum as an exercise ( http://forum.protonmania.com ) and I have my own little project, a wifi controller written using cake 1.2 ( radius,coova,mysql,linux,apache ) . Tried Sf, but cameback to cake..due to a lot of reasons.

Thomas said on October 17, 2008

I came pretty far in building my own CMS. I still got lots of ideas (many of them are custom plugins to make the whole thing more flexible and user friendly).

The problem is: I can't find the time to work it all out and build it. When a client needs something, it's sometime attractive for me to do it quick and dirty because there's just no time to build a universal solution.

Philippe Rathe said on October 24, 2008

To boost your JavaScript skill, nothing is better than trying to program a tetris game. You have to deal with many thing in order to program a game. I had a lot of fun doing it, even if it's not finished, it's still playable.

Anonymous said on November 03, 2008

Joomla has saved my hide so many times, but I personally prefer Drupal. Has anyone here used Elgg? Open source is good, but I would eventually like to build my own bloat free proprietary CRM, CMS and E-Commerce platforms to use on my clients sites. There is never enough time!!!

George said on November 23, 2008

Doing a weblog engine is good for exercising your web development skills.

John Lozo said on January 09, 2009

I am currently working on a blogging engine and I must say it is quite a challenge. I am thinking of switching on to CMS, or some combination of both. But I must agree with you that personal projects are just the best!

Sherry said on January 16, 2009

Excuse me. An idea is salvation by imagination.
I am from China and learning to read in English, tell me whether I wrote the following sentence: "First paragraphopen with the obligatory thank you and include how you enjoyed the meeting."

With respect ;-), Sherry.

Widliftingoli said on February 16, 2009

http://krser.com

urist said on February 17, 2009

Открытие и закрытие фирм, за 10000 рублей, Готовые фирмы за 29 000 рублей с директором и счётом в банке. Ликвидация организаций за 10000 рублей, получение извещений из Фондов, юридические адреса, продление, почтовое обслуживание, выписок из налоговой инспекции. Обналичивание, обналичка денег, получение денег наличными нашими директорами. Бесплатный выезд специалиста к Вам в офис. Обналичка (обналичивание) - реализация различного рода финансовых схем, направленных...
Деньги перечисляются посредством денежного перевода либо на карточный счет, который легко обналичить. Такая схема также является вполне законной и удобоваримой.
В нашем деле главное не цена, а надежность отношений.
Контактный телефон: 8-962-977-3711 Максим (Ваш регистратор)
Email: inmir@bk.ru
аська: 373336644

SeeniaUnserce said on February 17, 2009

http://nwser.com

Thitmedeemy said on February 18, 2009

http://utser.com

Calfeessfeede said on February 19, 2009

http://ortli.com

diborerce said on February 19, 2009

Can I restrict dating giant fib that's not undeniably internet related? I was at the exposed library one morning go in February checking my email (the days premier I had a computer at intimate with) and the neat guy sitting next to me started talking to me. He asked for my phone billion and asked if he could indulge me out sometime.

coolstUff</div>

Nataha said on February 19, 2009

Поздравляю всех мальчиков с 23 февраля! У Вас отичный сайт!

Мое поздравление - не звук пустой!
Мое поздравление - свято!
Оно для Вас - наш бывший герой,
Защитник наш в форме солдата!
И пусть пусть завоеванный мир хранит,
Святую свою безмятежность,
И пусть все люди, весь мир Вам дарит:
Любовь, признательность и нежность!

Sorry, comments are closed for this post. If you have any further questions or comments, feel free to send them to me directly.