Engage

Posted in:

I am a full-time consultant who is available to engage with clients remotely or onsite anywhere in the world (I currently hold dual-citizenship between Australia and the United States).

To discuss your specific needs, please call me on +1.650.336.5877, or email me at craig@craigsdickson.com, or use this Contact form, or download a copy of my resume from this page.

The following is an overview of the services I provide to clients:

Software Development Process Improvement

  • Coaching for Agile process evaluation, adoption or improvement, including Scrum, Lean, Kanban and Extreme Programming (XP)
  • Definition, refinement and documentation of team processes and practices
  • Definition of Quality Assurance and Quality Control standards
  • Integration of defect tracking systems with other tools and processes
  • Engagement with customers and requirements elicitation

Software Development Team Management

  • Job Description authoring
  • Salary range and benefits package definition
  • New candidate acquisition and screening
  • Team workspace design and office space evaluation
  • Skills assessment of existing resources
  • Collaboration strategies for teams

Vendor Management

  • New vendor discovery and screening
  • Vendor proposal reviews
  • Offshore vendor management, including onsite visits and reviews
  • One throat to choke multiple vendor management

Software Configuration Management (SCM)

  • Introduction of an SCM system to teams not already using one (Subversion, Git, CVS etc)
  • Subversion and CVS training
  • Subversion and CVS server installation and configuration
  • SCM process definition and documentation, including branching and merging processes
  • SCM system migration, particularly CVS to Subversion

Build Management

  • Implementation of Apache Maven and Apache Ant based build systems
  • Automation of builds, particularly in relation to a Continuous Integration system like CruiseControl or Hudson
  • Management and versioning of produced code artifacts, particularly in relation to an Artifact Repository like Nexus or Artifactory
  • Release numbering strategies and Alpha and Beta customer release programs

Software Architecture & Design

  • Enterprise-level system architecture definition, existing architecture reviews
  • New database design and existing database design review
  • Formal UML based architecture definition

Enterprise Java Development

  • Specialist in full-stack JavaEE development
  • Public API design and documentation for ISVs
  • Web service development and integration
  • Code reviews and performance tuning
  • Service Oriented Architecture (SOA) design and implementation

Web Development

  • HTML, JavaScript and CSS development
  • Integration of AJAX style JavaScript libraries including GWT, JQuery and ExtJS
  • Integration of Adobe Flash and Flex components

Automated Testing Strategies

  • Introduction of tools like JUnit and Sellenium to teams that currently do not do any automated testing
  • Integration of tests into automated build scripts and generation of metrics
  • Static analysis of codebase quality

Mobile Development

  • iPhone application design and development, specializing in integration to JavaEE based back ends
  • Web based mobile development

Social Media Strategy

  • Specializing in small to medium business that do not have dedicated in house Social Media resources
  • Evaluation of current Social Media presence
  • Recommendations for Social Media platforms based on particular business needs and goals
  • Evaluation of Location based services in relation to business needs and goals

Once again, to discuss your needs and to find out how I can help you, please contact me by phone on +1.650.336.5877, by email at craig@craigsdickson.com, or simply use this Contact form. If you would like more detailed information regarding my experience and qualifications, you can download a current copy of my professional resume from this page.

Why Automated Testing is Important – Part 2

Posted in: Software Development Best Practices

In Part 1 of this series I described the characteristics that make up a good Automated Test. Here in Part 2 of this series I will explore all of the benefits you will enjoy by creating those good tests and why the time spent on making good tests is a no-brainer investment.
Continue reading »

Why Automated Testing is Important – Part 1

Posted in: Software Development Best Practices

The adoption of Automated Testing strategies and tools, both in Agile and traditional teams, has been patchy – some teams and communities have embraced it, but many organizations still perceive it as a burden that just slows down development. Those that see the writing and execution of tests as an additional, costly and separate task from development have missed seeing some of the main benefits of an expertly manicured test suite.
Continue reading »

Scrum Anti-pattern: Outlier Pigs

Posted in: Software Development Team Leadership

In the Pig and Chicken analogy for Scrum participants (Jeff Sutherland explains Pigs & Chickens), the Pig is the one who is required to make the biggest commitment and put his proverbial skin in the game. For the Pig, it is an all or nothing proposition. They either meet their commitment or they do not, there is no gray area. However, many teams fail to get this level of commitment from their Pigs, or don’t even ask for it in the first place. This is the genesis of the Outlier Pig.
Continue reading »

Coding Standards – Quality From The Ground Up

Posted in: Software Development Best Practices

Coding styles are THE religious debate of the Software Engineering industry. Everyone has an opinion, but no one has an iron clad argument as to why their ideas are better than someone else’s.

It doesn’t matter what language you write your code in or what company your work for or even what open source project you contribute too, the topic of coding styles will sooner or later raise its head. The debate can range from the banal, like which line the curly brace goes on, to the overly subjective, like how to name variables.

In the end most of the decision points are pretty subjective and it is somewhat irrelevant what you choose, as long as everyone agrees and you are consistent. But don’t be mistaken, a consistent coding style is an important consideration on any project, from the solo developer to the multi-national team.
Continue reading »

JavaOne 2009 – (Mostly) Important Questions

Posted in: Enterprise Java

JavaOne 2009 starts in 5 days. Here is a list of questions I am looking forward to finding out the answers too.

Has Hudson Killed CruiseControl?
Seems like it has based on the number of mentions of Hudson vs CruiseControl in relation to the content at JavaOne. I lost my interest in CruiseControl when ThoughtWorks spun a for-profit version out of it. The only company I have seen be successful at this strategy is JBoss/RedHat where they develop the open-source version first and then roll the for-profit version out of that. The other times I have seen this attempted, all of the effort goes into the for-profit version and the open-source version ceases to progress. There is something fundamental about that 2nd pattern that just smells bad and doesn’t really seem to be in the spirit of open-source.

What Will Be The Volume Of The Twitter Noise Coming From Inside The Conference?
I have been tracking the hashed keywords related to the conference for a couple of weeks now. The volume has been slowly increasing and took a big jump on Tuesday morning when everyone got back from the long weekend in the US. I expect it to keep building up until Tuesday morning, but then what? Does it slow down because everyone is busy, or does it kick into a whole new gear and my trusty Twitterberry will just meltdown in the middle of the opening keynote?

Also curious to see what ad-hoc social activities get incubated in the Twitterverse during the conference?

Will AJAX Presentations Be THE Place To Be Seen For A 3rd Year Running?
The last 2 years have seen crazy interest in anything AJAX related. With Ben Galbraith and Dion Almer (spelling from memory there) being the focal point in their always entertaining presentations. But it feels a little like AJAX is getting to be slightly old news, at least in this forum.

My guess is that anything cloud related is going to be the hip place to be seen this year.

What Will The Oracle Presence Be?
AFAIK, the Oracle/Sun deal has not gone through yet, so technically Sun is still an independent entity. But of course I am also not naive enough to think Oracle won’t be pushing to start getting their hands on the “goods” at this conference. Will there be an Oracle presence in the keynotes that are traditionally Sun’s (the 2 on Tuesday and the 1 Friday morning)? What about signage around the conference? Oracle always has a booth in the pavilion, but will it be bigger, better positioned etc. this year?

What Will The Reaction To The Microsoft Keynote Be?
So the Twittervese exploded earlier this week when it was announced Microsoft will be presenting the Thursday morning keynote. Anyone who has been playing with Java long enough knows that Microsoft has not really been Java’s best friend. So, will the Java community accept Microsoft on the main stage? It would be nice to think that there will be some passionate reaction, either outrageous clapping or hateful booing, whatever, as long as there is some definitive reaction I will be happy. I fear the Java faithful might not be the kind to wear their hearts on their sleeves quite that much though.

Does the Oracle deal have something to do with Microsoft’s presence? Why does Oracle not have a keynote instead? Curious indeed.

Will Jonathon Schwartz Look As Uncomfortable And Awkward As Usual?
I will admit upfront that I am a Scott McNealy fan. He was passionate, and engaging to listen to on a stage. I was not happy when he was ousted from the top of Sun.

But even if I temper my anger over that situation, can anyone really be interested in listening to Schwartz talk? His stage presence is awful and he is robotic in his delivery of obviously scripted lines when guests are on stage. And don’t get me started on the pony tail, sport coat and jeans look! Bring back McNealy for the last one please!!!!

Will James Gosling’s Toy Show Seem Overly Long And Desperate Again?
I have a lot to thank James Gosling for. Most of my career is based on the technology he invented. I would like to have a beer with him at some point no doubt. But man, he is only marginally better than Schwartz on stage.

And I do not really understand the point of the Toy Show in the Friday morning keynote. You are at THE Java conference, and so the audience has self selected itself as resoundingly pro-Java. So why do we need a 3 hour carnival of Java applications trying to prove to us that Java is cool. We already think it is cool, that is why we are there. A lot of it just feels like they are pleading with us to please, please keep thinking Java is cool for another year until the next conference.

Will The Lunch Lines Be Under Control?
Getting your “free” lunch at JavaOne is an exercise in forgoing your basic right to not be hearded like livestock and yelled at by over zealous minimum wage event staff. It is like they are surprised by the number of people that show up for lunch each day, like there was no way they could possibly have guesstimated how many people might want to eat that day. Seriously, it is your last chance to get it right, please make an effort.

Will It Be Crazy Cold in Yerba Buena Gardens on Thursday Night Again?
Why is the Thursday night party outside now? I can’t possibly imagine it is much cheaper is it? It is San Francisco, it is cold on the hottest day of the year. I froze my ass off last year. The long range weather forecast looks like we are in for the same again.

Will The Bookstore Be Given More Space?
Doubt it. There is a whole convention center, and the bookstore gets jammed in a 10 by 30 square. Why? Why do you hate people who like to read?

Will Enough People Use me As A Reference So I Can Get The Better Swag?
I know 3 people who did, I think I need 2 more. I will even buy you a beer. My number is W1302019. Go ahead and earn yourself some karma points.

Why are the A’s and Giants both playing away all week?
A big boo to the MLB for having both teams out of town this week. It has become somewhat of a tradition for me to take my team to the baseball during JavaOne and you have destroyed that cherished pastime. Shame on you Bud Selig.

See you in San Francisco!

Get it right the first time – there is no going back!

Posted in: Software Development Best Practices

How many times have you been writing a piece of code and thought “Oh, that piece is going to be tough, I will put a dumb version in for the moment and come back and put the complete version later”. Probably often. Particularly if it is near the end of the project.

Usually this choice seems pretty benign, in fact it is second nature to most developers. But what are the issues with this approach?

The reality is that very few people ever get to go back and fix poorly written code. Even with the hype on refactoring these days, it really is not a widely spread practice (yet). Projects get pushed out the door prematurely to meet the almighty deadline and the team moves straight on to either the next version or the next project. In fact it is probably fair to say that if you have a bunch of time to be refactoring (assuming you are working on a “traditional” project where refactoring is a luxury at the end and not an everyday occurrence) that you are probably on a downward spiral in terms of job security. Your team should be 100% committed all the time – any other situation is probably a bad sign.

So what’s an engineer to do? Simple, just get it right the first time.

Getting it right the first time is actually spectacularly difficult for a whole bunch of reasons, many of which the engineer has no control over. What seemed perfectly logical will at some point turn out to have been a total waste or miscalculation. This is one of the main reasons the agile community has evolved. Agileists accept that getting it right straight off the bat is difficult, so they follow practices that limit the possible wasted effort by getting feedback early and often so they can more quickly identify when things are going wrong and change course quickly.

If you don’t have the luxury of working in a progressive company where agile techniques are embraced, you really have to be able swallow some hard truths. The biggest of which is, be careful what code you write as it will likely end up in a production environment and once that happens you will likely be supporting it for a long time, perhaps years or even decades. I once read a story (I forget where, someone please let me know if this sounds familiar), about a development team that wanted to ensure they did not commit too early to any technology or give customers false impressions about how much progress had been made, so what they did was literally prototype the UI experience with the customer using cutout felt squares and pinned them to a board and rearranging them until the customer had what they needed.

So think about what you are writing, quality is something you need to worry about now, not just during the QA phase of your project. And quality is more than just whether the code meets the functional requirements without setting the computer on fire. Quality is everything from following your teams coding conventions and code documentation, to extensibility and maintainability.

So if you are in an agile team, realise that the higher the quality of the code base, the more value it has to your company and/or your customer. To that end, refactor mercilessly. If you work in a traditional team, before you write a single line of code, think to yourself “would I want to maintain this piece of code I am thinking of writing” because that is exactly what you will be doing if you write it.

Upgrades, upgrades, upgrades everywhere

Posted in: Enterprise Java

We are bombarded with hype about new versions of tools (like IDEs), or new libraries (like a new JDK), or a new software package (like a new object-relational mapper) all day long. But do we even need them? Can’t we basically create what we need to complete these mundane everyday tasks with the tools we already have? Aren’t most of these new releases just a way to sell new software licenses? Even when it comes to opensource software, upgrading to a new release has an inherent cost that needs to be weighed.

Here is a real world example. I am currently invovled in a project that involves upgrading the code base from JDK 1.4 to JDK 5 as part of the scope. My Engineer brain says that this upgrade is not only necessary, but just a plain good idea. There are some new language features in JDK 5 that we can take advantage of, so there is a benefit. But what are the costs? There is definitely a cost involved in doing an audit of the existing code and seeing if there is any code that is incompatible with the new JDK and putting together a plan and effort estimate to address those issues. There is also a optional cost of retrofitting the existing code to take advantage of the new language features (like enums for example). There is a learning curve cost. Perhaps there is a confusion cost for some junior engineers (I am thinking of the autoboxing feature). There is a cascading cost that we now also have to consider upgrading our application server environment to support the JDK 5 code.

So is all of this cost worth it? Will the benefits outweigh the risk and give the company some kind of ROI? I think in most cases, the answer is actually a defiinite yes.

Take the Java 5 upgrade example. Apart from the obvious technical benefits, there are a whole host of trickle down posiitve benefits. For starters, I think many companies underestimate the importance of keeping the geeks happy. Engineers like to use new tools and stay on the cutting edge. They like to be able to read articles in the latest journals and be able to apply the lessons learned straight away. Happy Engineers are good employees. They are more productive. They are more collaborative. They tend to not be looking quite so hard for that next dream job. They tend to be more forgiving of working conditions that might not be ideal. Engineers are creatures who like to learn and keep learning. Working on the same code and project and tools for year after year will be the death of an Engineer and you will end up with just mediocre Coders left on your team as the Engineers will be long gone.

So before you decide on your next upgrade, think about all of the costs involved, there are many hidden ones you may overlook at first glance. But then make sure you also think about all of the benefits (both short and long term). Not every upgrade will make sense, but I bet that a majority of them will.

Enough is Enough

Posted in: Software Development Best Practices, Software Development Team Leadership

I am personally interested in processes (or removing processes), practices and tools that allow software teams to deliver on time and with high quality on a consistent basis. Now that sounds pretty benign and uninspired. It sounds like a common goal that everyone in the software industry would share. Am I naive to think that it is more than reasonable to expect bug-free software? I know many people think that at a mathematical level, debugging many of today’s complex systems is simply unrealistic. But didn’t humans design the compilers and the IDEs and everything else related to software? Don’t we by definition control the monster? Are we really so smart as to have invented something that is beyond our own abilities and comprehension already? Is this the start of the rise of the machines? I choose to think not.

I do believe that bug-free software is the goal of every software project, even if it isn’t explicitly stated. The problem is the commitment to achieving that goal. Quality is often intangible, at least on the high end (low quality is often very tangible). And intangible things are easy to ignore or sacrifice. Quality often comes at the end of a software project and when deadlines are tight, it is the first to be discounted.

Here is the story of what finally inspired me to start this blog.

I was sitting at San Francisco airport, waiting to fly home. I live in southern California, so it is a short flight and American Airlines only uses little regional jets for the SFO to Orange County route. The flight was not full and the plane is small to start with. So passenger loading was done pretty quick and we were set for an on time take off. But we didn’t take off. We just sat there and sat there. Finally the stewardess announced over the intercom that the delay was because they had introduced a new automated system that week to do some of the paperwork required to be given a green light for take off. And that they were having some trouble getting it to work so they were going back to the manual system so we could get underway.

The most disturbing part of this was the exact words used by the stewardess during the announcement. She said “we are using a new computerized automated system and as with all new software there are some bugs and glitches”. I found that statement to be very profound.

My assumption is that an airline stewardess represents a typical layman user of software – in other words she has no specialized knowledge of how to write software or what it is like to work in a software team. She represents a typical user that only sees software from the end-user perspective. And from her perspective software usually has bugs. Usually has bugs. Can that be true? Over all of the software written and all the smart people that write it, does software still usually have bugs? At least anecdotally in my experience that does appear to be the case.

So I say to that, ENOUGH IS ENOUGH.

Software is now part of a lot of people’s every day lives. It is responsible for many tasks that if completed incorrectly could cost human lives. It is also responsible for many tasks that if completed incorrectly have a financial impact on companies and individuals. There is also a social impact from poor quality software. What was the cost in terms of lost time for the people sitting on that plane with me? What is 20 minutes worth? Time is the only non-renewable resource, so 20 minutes is actually priceless, its value is immeasurable.

Every time you relax your definition of quality you are failing as a software engineer. You are contributing to the continued and currently deserved poor perception that people have of the software industry. And ironically, you are also immediately failing the customer who is probably the one pushing you to do the relaxing. It is a basic, fundamental principle of software development that a relentless pursuit of delivering quality software is actually the shortest and quickest path to delivering a project.

Better the devil you know

Posted in: Software Development Best Practices, Software Development Team Leadership

It seems nearly everyday a new technology or pattern or paradigm bursts onto the scene. Lately we have seen things like AJAX and Ruby and other web-oriented technologies getting a lot of buzz and plenty of people jumping on the bandwagon. And don’t let me get started on SOA!

But what about the trusted tools we already know. Do we need to be ever pursuing the mastery of these new technologies and applying them to each new project and retroactively applying them to existing (already functioning) projects? What value does that bring? It really only makes sense if the benefits of the new technology outweigh the costs of adoption – these costs can be higher than you think.

To illustrate the point, lets take a team of engineers working for an internal IT shop. They have worked to identify a core set of tools and technologies that they are going to specialize in. They have recruited for those skills and their training dollars have been spent to improve those skills. In addition they have wisely created coding style guides, they have documented their own best practices they have chosen to follow, and best of all they have a continuous integration server set up, that is building and testing their code as fast as they can commit it. All in all, they are quite comfortable in their chosen domain and are working to maintain their skills appropriately.

Now for some reason they are asked to consider technology “X”. This could happen for a bunch of reasons – maybe somebody writing the functional requirements was over zealous and mentioned it in those documents, or a customer has heard of “X” and simply wants to have their project developed with it for no other particular reason except that they have heard of it and want to “contribute” to the process.

Assuming the team is a reasonably mature team with a good mix of senior and junior talent and have a grasp of fundamental engineering principles (quite rare possibly?) then what is the cost of using “X” on the new project instead of their existing tools?

As in most organizations that practice some form of waterfall process, they start by asking the engineering team for a detailed estimate of how long the new project will take to implement. Ignoring the fact that this is a gross error to attempt such an estimate at this stage of the project, the team dutifully works to create the estimate. But now they are faced with the problem that they are not really sure how long it takes to do things with “X”. They are comfortable with their current set of tools and could probably come up with a good estimate if given enough time, but now they are really going to struggle. So they scramble to find some documents, but of course “X” is the new sexy technology with a lot of buzz but no solid documentation or best practices defined. The team’s only true choice (assuming the estimate is needed quickly) is to SWAG it and then likely pad the SWAG to allow for the unfamiliarity of “X”.

So the first issue we have is a wildly unreliable effort estimate due to a lack of experience and confidence with “X”.

Now its time for design. But of course the team has no inherent knowledge of “X” so they get nothing for free in the design phase and have to start from scratch on every detail. So the design phase progresses and the team is able to make some progress, but there are some nagging questions that the team simply cannot find answers for. The team identifies these issues and decides that the only real way to resolve them is to engage some external help from the only people who know “X” – the vendor that just launched it. The vendor is of course more than happy to help, but since they are in great demand (because they are the only ones with any skills in this area), their rates are astronomical. The team reluctantly engages the external resources and embarks on a Proof Of Concept to help finish the design phase.

So the second issue we have is a design phase that takes an incredible amount of time, most of which can be considered more “learning” time than design time and we also have a huge cost outlay for professional services.

Time to implement.

The team is trying to be as agile as possible within the bounds of their company’s tollerance so they are quite commited to unit testing and continuous integration, but pair programming and other practices are still just “crazy talk”. Unfortunately “X” dosen’t have a good toolkit for writing unit tests. The vendor says they are working on it, but it is not on the release calendar yet. Looks like unit testing is out. The team starts implementing, but their preferred IDE dosent support “X” so now they have to run 2 large IDE applications on their (already underpowered) machines at the same time so they can creat the “X” parts, while still having access to the rest of their projects. Additionally, the team has been committed to creating repeatable builds and strict release processes for a while now since some code slipped out to production a while back without being tested or put into there code versioning system. Technology “X” builds fine within the customized IDE the vendor has provided, but there is no support for compiling from other command-line based build tools. So the team assigns resources to write the necessary build tool plugins so that they can appropriately build, tag and release the project.

There is a lot of additional overhead in the implementation phase when using a brand new technology. Over time these overheads reduce on each subsequent project, but the cost on the first few is quite significant. The lack of integrated tools really does hamper the team and the costs incurred because of the extended development time will add up quickly.

Eventually the team finishes implementing, but since no automated unit or integration tests have been running, the quality is questionable. Never the less, the QA team comes along and wants to test the project. But once again, “X” is so new that there are no tools for the QA team either. But even if there were tools available, the QA team would have never used them, so there would be a training and ramp up cost involved. So QA is going to be a manual process. As a result, some of the regression testing is not applied consistently and a lot of bugs slip through the cracks and make it to production.

The cost here is of course an extended QA time line. But possibly more costly is the low quality of the end product.

The project finally goes to production, but of course there is the immediate flurry of bugs and feature enhancement requests that are associated with a 1.0 release.

So the team starts again with the requirements gathering phase. But when it comes to the design and implementation phase, it turns out that in hindsight the code wasn’t really architected as well as it could have been now that the team knows more about “X”. So it is difficult to add the new features because of this. They hack the new features in, and so begins the downward spiral of building more and more features on top of an constantly degrading base. Technical debt increases exponentially.

This cycle of adding features and then fixing the bugs the new features introduced continues for a while. But now 2 of the team on the project are leaving for greener pastures (its surprising they lasted this long really). So the HR person comes by your office and gets a detailed job description from you, which includes all of the skills from the other technologies but now also includes “X”. The HR person not knowing any better says thank you and goes off to do the best keyword matching process they can.

A day or two latter they come back looking frustrated. They are having trouble finding anyone with the mandatory skills you described. Turns out that the two skill sets you must now support are really not common bedfellows and so recruiting is going to be an issue. You really only have one choice – recruit for one skill and spend the money and time to train them on the other.

So here is an ongoing maintenance cost that will be higher than before. Recruiting for skills that don’t usually show up together can be a big issue. If you are lucky enough to find people with the skills, they are likely to be higher priced just because they have that uncommon combination.

So before you choose to adopt the latest sexy technology because all of the weekly e-newsletters you get are mentioning it, think about what it is truly going to cost your team and company. Is it worth it? I propose that in the majority of cases, it is not.