A software development web blog

You are on your way to visit your Grandma, who lives at the end of the valley. It’s her birthday, and you want to give her the cakes you’ve made.

Between your house and her house, you have to cross 7 bridges, and as it goes in the land of make believe, there is a troll under every bridge! Each troll, quite rightly, insists that you pay a troll toll. Before you can cross their bridge, you have to give them half of the cakes you are carrying, but as they are kind trolls, they each give you back a single cake.

How many cakes do you have to leave home with to make sure that you arrive at Grandma’s with exactly 2 cakes?

Sometimes developers get stuck in one language. Luckily I bounced around in my early years, but for a good long time I was stuck in a php environment. It was so easy to get the next PHP gig, that I just kept doing it. Moving to Ruby on Rails for a new project at work has been really enlightening, and has reminded me of something.

I am a programmer, not a php developer.

I don’t even know why I kept taking jobs as a PHP developer – I never really even liked the language very much. I think I got lazy and it was a comfort zone.

As a software developer, breaking out of your comfort zone is really the hardest part about learning a new language.

I was always a fan of python and perl (And C some days), which just adds to my confusion when I look back and see how long I’ve been futzing about in PHP.

So anyway, we decided that Ruby would be a better language to use on a project at work. So I had to try to learn it. At first, I thought it was going to take the whole weekend – but it didn’t – thats partially the beauty of Ruby as a language, and partially as my Mom always said “Nothing is ever as bad as you think its gonna be. Now go clean the toilet.”

Learning Ruby was far from cleaning a toilet however – it was an enlightening experience that woke up certain areas of my mind that had been dormant for many years.

I guess it is a lot like learning a new speaking language.

The syntax might be different – but the structure is the same. You have to tell someone you’re going to lunch? All you need is a word for “I” (Me) , and a verb for going, so people don’t think you ARE lunch – and then the word for lunch itself. That really isn’t that hard. In the process you learn a little somthing about language itself that’s independent of the specifics of any particular language.

Same with programming. You know you will need a variable here, an array there, this can be a function, and that can be a class, you need to open a file, ok – theres a way to do that – the similarities are there, and might surprise you.

I came from basic (doesn’t count in my opinion, I was about 8-10 yrs old) to C – then to perl – then php and then python. So when it came to Ruby I was a fan, it’s a lot like perl and python.

But I have always thought – that you really become a “Programmer” when you learn your second language. It is then that you become familiar with the structures, and patterns and members of a programming language, and your brain suddenly gets a glimpse of the bigger picture. You realize that there is more than one way to do things – and you also have a lot of a-ha moments.

If you have only ever used PHP for example, you might know that when you make a class it acts in a certain way – it does certain things – and thats about it. You expect it to behave the way you have always known it to behave, and you don’t really think much more about it.

On your second language you have to figure out how to make a class all over again, and suddenly you start to see differences in the way its done. You also see the similarities. At that moment, your brain latches on to the “pattern” that is a class. Once you remove the differences between your class in one language and your class in the other language – whats left? A class.

The most basic – plain – stripped of the syntax that disguises it – class. In this form, you can finally get a handle on the fact that you are dealing with abstract ideas here, not syntax. Whats a variable anyway? its not $this or #{this} or @@this. Its a container that has a type, state, and can hold information.

This happens again and again as you discover how to open files, or grab a URL make a function or method, call it with parameters, etc.

Lets take my ridiculous example of going from basic to C. It even works there.

Yeah, BASIC. Like on a TRS-80. I remember a few things about basic. Line numbers. GOTO and GOSUB. Holy cow, it hurts to think about it now. When I was “liberated” by C, it was such a relief, and I didn’t need a book to tell me that goto and gosub were strange attempts at functions and loops. I do remember feeling a bit weird without my line numbers. But when I realized I wasn’t naked – but rather free – I started to see all the ways in which line numbers held me back.

I remember looking back at old basic programs I had written and thinking – “Wow, I could have used a private variable here in a function instead of needing to spawn several other vars to handle this loop” – Stuff like that.

This was 1995 or so and in the time since then, I have never been afraid of new languages, but rather excited each time I got to learn one. I did fall into the trap of spending too long on PHP – but during that time I was playing with python.

So go ahead. Pick up another language and run with it. If you are a PHP person, try ruby or python. If you use one of those Microsoft languages, try perl. (Also check out my favorite people over at ActiveState – they can hook you up with the tools)

Liberate yourself. Those who have done it know what I am talking about

So I kept seeing this in my web stats, as a search term ALOT of people are using to find this site.

Of course, they are probably finding my articles that mention cake php and the one about duplicate keys.

So I had to google it, to see what people REALLY want to know. Ends up its a joke.

I’m sure that while they may enjoy my articles, the articles are not what they really want in this case.

So here is the answer you are looking for, folks.

Why is a duplicate key like a small cake?

Because its a twinkie. (Twinkey)

Sigh.

Ben Collins-Sussman and Brian Fitzpatrick are the originators of Subversion (svn) and now both work for google.
If you haven’t already seen this, you need to. They are discussing that there is no “Genius Programmer”, and how working with others on a team (such as in your workplace, or in an open source environment) is the best way to go.

This is a great video, its about an hour, but completely engaging. Ben and Brian work really well together and they are always fun to watch.

So, let me take the time to say, that there are alot of reasons why this might be nothing – but that’s not the point of this article! The point is to say – take time to play with your skills! Do things that are simply fun and will teach you along the way.

I saw a tv show the other day – something on the Discovery channel that I was barely watching. They were talking about the Fibonacci series – numbers that are made by adding the preceding two numbers together. They are:

0,1,1,2,3,5,8,13, .. etc

As you can see, they are made by adding the two previous numbers. 0+1 = 1, 1+1 = 2, 1+2 = 3, etc.

Well I wrote a quick program to generate, then graph the results of the numbers, and got a pretty nice curve on the Fibonacci numbers themselves:

As you can see, its a flatline until it trends up exponentially at the end. This is to be expected with a mathematical equation that snowballs like the Fibonacci series.

Then I tried a few other arbitrary experiments – things  like adding all the digits in the number and graphing that – or even taking the sum of the numbers divided by the Fibonacci number itself, and didn’t really get many more interesting results.

Then I started thinking about 9’s and all the magical mathematical fun you can have with them, specifically digital roots.

Digital roots are when you keep adding up the numbers until you arrive at a single digit, that’s the digital root. So for instance:

Take the number 348. 3+4+8 = 15. Still not a single digit, so you do it again. 1+5 =6. That’s a single digit so you know the digital root of 348 is 6.

If you have a calculator, its easier. Just divide the number by 9 and take the first digit of the remainder. 348 / 9 = 38.66, and the first digit of the remainder is 6 – same thing.

In code (since that’s what this blog is about anyway) for instance, you can do it with the modulus operator. The modulus operator is the % (percent) sign. You can do it like this:

$digital_root = $number % 9;

So, I started taking the digital root of the Fibonacci numbers, and I got some strange results. First of all, the numbers repeat every 24 times – it just starts over again, which we can tell is going to be some sort of interesting pattern on the graph.

It wasn’t very interesting on the line graph, but then I decided to try a scatter graph, and it was kind of cool:

It looks a bit like DNA! In fact, DNA is 22-26 angstroms in length, which would fit the pattern we found of this one repeating every 24 times.

This may be nothing – and it probably is. But we learned alot in a very short time armed only with php (and Jpgraph) and a bit of creative thought. Who knows what new patterns you could find if you mapped out the points on this graph, and applied them to a real DNA model, and started comparing numbers?

So this is what I encourage you all to do – prototype some new and creative ideas! This kind of experimenting is possible now with computers (and those of us who know how to program them) – and it was much harder back in the day when mathematicians and physicists had to do it all by hand.

Also, it is this kind of “playing” and “experimenting” and “question asking” that has lead to some really great discoveries!

I wrote this article because web programmers, sometimes forget they are programmers at heart. Take away the web every now and then, and see what you can discover with what’s left!

If you are interested in reading more about Fibonacci numbers in DNA, this is a pretty good read.

Since this blog is primarily about the LAMP environment (php) being the key language I thought I would address a problem thats pretty common among a good percentage of php coders – and that is the tendency to be hackers.

I don’t mean you are a hacker if you are a PHP developer. Not at all.

I mean that a good percentage of folks who LEARNED the fine art of programming by becoming LAMP developers often lack the high level structural thinking that some folks from other backgrounds have.

The reason for this is, PHP is / was the biggest “web programming” language that was easy to learn for a beginner, with ample webhosting, mountains of documentation and help on the web, and therefore drew in a bunch of beginning programmers. Nothing wrong with that. Another reason would be that PHP was not always an OOP language – this was introduced later on, so all the old PHP guys either had to learn it, or just didn’t.

Myself, having come from a C, C++, perl background, was always trained out of necessity to think something through before going into the code.

But in hiring, interviewing, and eventually working with php developers, I consistently am surprised how many of them who’s first reaction is to hack their way to a solution instead of coming up with a graceful end product.

They often lack knowledge of architecture types, frameworks, and object oriented theory.

I shouldn’t be surprised though. A lot of them learned themselves through trial and error, and never had a mentor who had been there before, or their mentor if they had one, learned the same way they had – trial and error.

And PHP is SO forgiving, it really doesn’t force you to learn a framework, or a structure, or even one way to do things. On one hand this is a wonderful thing about PHP – the fact that its easy to learn and you can knock out an application fairly easily – many great minds – and their applications saw the light of day with PHP – people who had wonderful ideas, but didn’t have the time to become “application developers” first.

So lets quit dinging these guys for their style (or lack of…) and help them out.

So lets say you have 3 employees, of course not counting yourself.

Sammy is a junior developer. He knows php, decent with MySQL but not great. He can write functions, but OOP confuses him. No framework knowledge at all.

Johnny is a mid level guy. He’s good with classes, MySQL, and he’s played around with Cake PHP and he thinks its really cool but he keeps breaking the rules because he’s not used to MVC or a framework, and – well – because he can , and sometimes its the easiest way to the solution for him.

Tony is a skilled guy, he’s all about objects, classes, frameworks, he’s more of a high level thinker and he’s the best guy you have. If you died in a bizarre gardening accident tomorrow, Tony would get your job.

Sammy and Johnny need to be willing to learn, and Tony needs to be willing to help you teach. You might have 10 Sammys, 5 Johnnys, and 3 Tonys – it doesn’t matter. The first step is to identify which of your employees can help you teach, and which ones need to learn.

Requirements:

1 – The developers must be willing to learn and change.
Sell them on the benefits. Sell them on the ease and time saving factors. Sell them of convention over configuration. The biggest resistance you will meet in a developer is the fear of the unknown – the fear of not having the answer in a critical moment. Make them realize that what you are teaching them isn’t as far as it sounds from what they are already doing. Its organizing their code – encapsulating it, making it more modular, eliminating drama, stress, and spaghetti code.

2 – Your company must be behind you in the fact that learning is paramount.
First off – don’t take too much of the company time for your teaching sessions. Keep it to the bare minimum, and leave some of the details for on-the-job learning. The minute you propose a two week hiatus in the production schedule or a 3 hour per day training class for all the developers, the management will jump ship on the idea, leaving you right back where you started. Companies vary, and so do managers, hopefully your company understands the value of education, (especially – and sell them on this – a free one) for their employees.

The selling points are many – much less time for development. MVC makes development very fast! Much less time for maintenance. With the convention over configuration, even a developer who has never been in a particular piece of code should be able to adjust very quickly to his new environment – because its not a new environment after all! Sell them on all this, it’s all true. It is true that a smaller investment in training and education will pay off handsomely, and not in a few years – more like a few weeks or months!

Step One: Introduce MVC, The Cake framework, and some project planning basics
For this part, don’t worry about objects and classes – that will come – do a quick overview of OOP so they understand, but don’t stall here. Your junior developers can work closer to the framework, and your more advanced ones can work with the objects and classes. Have a weekly class where you first introduce MVC, and then do some examples in Cake. This keeps them in their comfort zone with the code, freeing up mental space for the higher concepts.

I would see it something like this:

Class One: MVC Overview
Class Two: OOP Overview, the very basics
Class Three: Introduce Cake and Tie it all together
Class Four: Introduce project planning charts like model and class diagrams, but don’t go full blown UML – they will die of boredom.
Class Five: Plan out a sample sample application, which will be done in Cake, using what they learned in Class Four.
Class Six: As a team, get the application framed out in Cake, show them how easy it is.

We are now only six weeks into this – six one hour courses. Adjust this as you need. You might find that some of them try to break Cake’s rules and go back to procedural coding, as this is possible with Cake. If this happens and becomes a problem, try Step Two:

Step Two: Force them by introducing another language – Ruby – with Ruby on Rails.
a – this forces your Johnnys to learn the system – all their dirty little tricks in php are gone, they have to play by the rules.
a – this also keeps them learning together, Johnny is now no better than Sammy, they can help each other out. You might see Sammy surpass Johnny here.
b – develop a simple simple app with them to show them the ropes

Be careful with Step Two – you might end up convincing everyone to stick with Rails!

I have to thank my friend Russ for getting me thinking about this stuff again!

I wanted to recommend a great book that I’ve read two times now. It wasn’t so complicated that I had to read it twice, its just a great book, and it’s fun to read if you work in the field that I assume you do if you are reading this site

The book is called “Release It!” by Micheal T. Nygard.

The book deals with launching and releasing a product that is production ready – really ready.

Sometimes we get caught up in processes, and how things “Should be done” and we forget to address one of the most critical aspects of launching a software product – which is to ask the question: “Will this stand up to a sudden hit from a web link? Will it survive 100,000 hits in an hour?”

Especially with the internet today, even the little guy has the opportunity to put up a webpage that might attract enough attention to crash itself.

With popular web services like Digg, Stumbleupon, Yahoo! Buzz, etc., this could even happen accidentally.

Or at work – I’m sure your boss and your marketing department have big plans for the project you are working on – if they do their job right – and send tons of traffic to your app, you better hope you did your part right – the last thing you want is for your software to barf at that critical moment, where thousands of people are trying to use/buy/read your product.

The boss won’t be happy!

I have experienced this a few times myself. Software that was written to do a job once – where the developer forgot to address the x 100,000 factor – the software needing the ability to do the job 100,000 times in a short period of time. I was the lucky guy who got to go into the software and try to figure out what was going wrong, and fix it as soon as possible, and then audit the rest of the system to make sure it wouldn’t happen again.

Those experiences made me cautious – but this book taught me how to think differently.

Release it won’t tell you exactly HOW to make sure your project is ready for whatever beatings your marketing team has planned for it, but rather, it will shift your thinking toward architecting a product that will handle real world problems that it might face. The details, of course are up to you.

Each section of the book starts with a real life case study – a horror story for any developer or architect. Michael Nygard looks into each case, and discusses what happened, why it happened, how it was resolved at the time, and what could have been done to prevent it.

Regardless of your platform, if you are a developer or software architect, you really need to check out this book!

Here is a link to Amazon.

Release It!: Design and Deploy Production-Ready Software (Pragmatic Programmers)

MySQL : Insert and replace and ‘on duplicate key’ when and how to use them.

Lets consider the problem of a counter, which is the most frequent dilemma that results in this question being asked.

We have a website with articles, and each article can be rated. A user can “vote” for an article, much like digg or yahoo buzz.

So you create a simple database.

CREATE TABLE `votes` (
 	 `id` int(10) NOT NULL auto_increment,
 	 `article_id` int(11) default NULL,
 	 `vote_count` int(11) default NULL,
 	 `vote_pct` float(5,2) NOT NULL,
  	 PRIMARY KEY  (`id`),
 	 UNIQUE KEY `article_id` (`article_id`)
);

Simple enough right? When an article gets a vote, you simply increment the counter (vote_count) by one. Lets say later, a cron script comes through the database and tallies up what percentage of the votes that article has, and stores it in vote_pct.

Suddenly, you realize you have a small coding issue. The first time the article in question gets a vote, you need to insert a record with the article_id, and a value of ‘1′ for the vote_count. The second vote, however, you cant simply do another insert, you need to do an update to increment that article’s vote_count by one.

Lets say the article in question has a unique id of “55″ (article_id).

The clumsy way would be (and I have seen this done many times!):

// figure out if this article has ever gotten a vote, by seeing if its in the table already
$find_query = "select count(0) as count from votes where article_id =55";
// then after fetching the result, do an if to either insert or update


if ($result['count'] >0){
	// we are updating, and incrementing
	$sql = "update votes set vote_count = (vote_count +1) where article_id ='55'";
}else{
	// we are inserting and setting vote_count to 1
	$sql = "insert into votes set article_id ='55', vote_count =1";
}
// then run the query
mysql_query($sql)

Waaaay too much code!

At first glance, you might consider using replace. But you can’t do this, because then you would still need two queries, one to see how many votes the article has, and another for the replace. Also you would need all the data again (like the vote_pct). To understand why, you have to understand how replace works.

How replace works:

It either inserts, just like an insert statement, or it deletes, then inserts, if it finds a unique or primary key that matches. So you would lose any data that you didn’t specify in your replace statement. Our table only has 4 columns, so this isn’t a big deal, but imagine if you had other data in the table, it would be practically impossible to do this without first selecting all the data, then replacing it with the incremented counter. Still, too much work.

The wrong way:

replace into votes set article_id=55, vote_count = (vote_count +1)

This would silently fail, in other words:

In the case of a new record, NULL +1 still equals NULL so your count of one wouldn’t be recorded.

Also, any other data outside of article_id and vote_count would be blown away, including vote_pct and your ID which may be used elsewhere in the software, would be set to the next available increment value, in other words, it would keep changing. Less than helpful if you use this data to keep track of anything.

Enter ‘on duplicate key’:

I am always surprised how many great developers don’t know about ‘ON DUPLICATE KEY’. Its been around for a long time!

It would go a little something like this:

INSERT INTO votes
SET article_id =55,
vote_count =1
 ON DUPLICATE KEY UPDATE vote_count = ( vote_count +1 )

Here, it would try the insert, and if it found the unique key (article_id) was already there, it would update it and just increment the vote_count, and leave the rest of the data intact.

If it didn’t find the article_id of 55, it would go ahead and insert it.

Of course for any of this to work properly. article_id would have to be an index, of the unique type (or your primary key).

So every day, whether I am at home or work, I have the same setup, and I often take it for granted.

Today, I read some articles while commuting on the train that made me wonder – what would I do if I had to start over and do the same job, but with all new tools? I would probably cry.

Of course I started doing this when none of this stuff was around, it was all simple text editors, command line, and so fourth. But these tools make life so much easier, they all deserve credit for getting me down the road each day.

Some of these tools are debatable, and open to personal preference. Also, I work on a mac, so this article is a bit mac-slanted.

But anyway – here is my list of must-have’s (in no particular order) that get allow me to work as fast and accurately as possible!

1 – Mac (Machine)
I am a mac person. I have keystrokes setup for everything, use the spotlight all the time, and touch my mouse as little as possible. In all reality the main reason I like working on a mac is because of the speed benefit – not that it runs faster than a PC (theres an argument waiting to happen) but the speed it allows ME to work at.

2 - Komodo Edit or IDE (Text Editor)
I came across Activestate one day around 2000 or so, when I was trying to find a way to get perl to run on a windows machine. I found ActivePerl – and later, Komodo.
After trying all the other editors out there – ALL of them, I kept coming back to Komodo because it does all of the things I like, and doesn’t do any of the things I don’t like. Code completion, brace balancing, extreme configureability, etc. Its easy to get, and install, plus has a good community of people writing plug-ins for it.

3 – Google (Search Engine)
C’mon, who doesn’t?

4 – Unfuddle (Project Management / Subversion)
Only about two years ago, a friend of mine told me about Unfuddle. At the time it was a good way for me to get a subversion repository cheap and quick without the hassle of setting my own up. The repository is only one benefit, however, its a complete project manager as well.
Milestones, Notebooks, Tickets (assignable to milestones), multiple users (for a decent price), internal messaging system, and more.
Theres a free account with limitations such as two users, but still with unlimited svn and git repositories. For my personal stuff, Unfuddle is indispensable!

5 – MySQL Workbench (MySQL interface)
I think I found this back around 1999 or 2000 and I think it was called MySQL Front. It was kinda crappy then and only dealt with issuing commands and such. Also my boss blew away a whole database worth of information by forgetting a where clause while using it back then.
If you don’t like phpmyadmin, or cant install it where you are, its a great interface to your DB. It has an administrator as well (formerly MySQL Administrator) so you can look at and modify server level stuff (pending your perms obviously).

6 – Firebug (Front end debugger)
Firebug has been a lifesaver over and over and over again. I could write (and I might) write a whole article about firebug. You can watch the http stream, debug js from the browser, debug, change, modify css from the browser. It does SO much, all I can say is, if you use firefox, GET THIS. Read a tutorial on all it can do, or bug me to write one.

7- Google Calendar (Dont forget stuff!)
Ok, so your head is so deep in code all day, and you probably have some kinda of corporate calendar for work stuff – but what about calling Mom on her birthday or picking up something for your spouse on the way home? I don’t like to mix personal events with work stuff, and I like google calendar because it is viewable everywhere. My cell phone, any computer with an internet connection, etc.
The coolest feature is this – I have it connected to my phone number, so I can send a txt message from my cell to googles SMS number, in pretty common english, such as “tomorrow at 6pm pick up kids” and it will add it to my calendar, no matter where I am. Also, I have it setup so it will email and txt me back prior to the appointment, so if my cell is in my coat pocket, I get the email, and if I’m not at a computer (ha!) i get the txt.
I know this is blurring the line and isn’t really a development tool, but it allows me to be at ease knowing I can dig in an concentrate, and will be reminded if I am coming close to forgetting anything personal.

8 – Wamp or Mamp (Lamp environments)
These are great, MAMP for Mac users, WAMP for Windows users. Basically an entire LAMP (Linux, Apache, Mysql, PHP) server system set up on your machine in no time flat. I use it for playing with code on the train or if I know I wont have internet somewhere.

9 – Reference sites I use all the time:
The subversion book
Cake PHP

As well as the ones listed to the right in my links section

10 – Cake PHP (Framework)
I do most of the architecture for a new project on paper, and then transfer it to whatever format my stakeholders / employees are expecting. I guess I’m old school in that way. There wasn’t any software I really liked when I first starting deploying projects that required higher level planning, so I got used to pencil and paper. Plus I like scribbling and drawing, its fun!
But for fast prototyping, and even deployment of some projects, Cake PHP is the fast road to getting it done for me. I wouldn’t say that I would launch every php project with Cake, but, its very useful for some, and for almost anything that’s a prototype.
It forces you to keep an eye on the big picture, and of course saves you and any other developer who looks at it later some time with the famous “Convention over Configuration” Paradigm.
Once you learn the ins and outs, you can knock out a moderately complex prototype (minus error trapping and nice things like that) in just a few days. I was introduced to Cake by the same guy who introduced me to Unfuddle (Thanks Tony!). At the time I was considering Rails or attempting to write my own framework, which I am still thinking about!

So thats it. I probably missed some, but as far as a snapshot of what I do daily at this time, these are the tools I find most beneficial. I hope this helps some of you!

Tim

This one is pretty easy, but it comes up alot.

This would be a situation wherefor instance a certain configuration file has to be specific to its environment, say in a distributed server software environment. Commit all the code, but don’t commit the config and overwrite everyone’s config file each time.

svn delete --keep-local file_name