Continuous Deployment at IMVU: Doing the impossible fifty times a day. « Timothy Fitz
http://timothyfitz.wordpress.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/

Continuous Deployment isn’t just an abstract theory. At IMVU it’s a core part of our culture to ship. It’s also not a new technique here, we’ve been practicing continuous deployment for years; far longer than I’ve been a member of this startup.

unknown terms: tdd, rsync, symlink, schema learned new defn: integrate need to learn big o notat... later. need to sleep

"On average we deploy new code fifty times a day. So what magic happens in our test suite that allows us to skip having a manual Quality Assurance step in our deploy process? The magic is in the scope, scale and thoroughness. It’s a thousand test files and counting. 4.4 machine hours of automated tests to be exact. "

Need to gather the linked PDFs sometime

agile

eployment script. At IMVU, we built a serious deployment script that incrementally deploys software machine-by-machine and monitors the health of the cluster and the business along the way so that it can do a fast-revert if something looks amiss. We call it a cluster immune system. But we didn’t start out that way. In fact, attempting to build a complex deployment system like that from scratch is a bad idea.

development deployment programming testing agile

Experiences deploying a large-scale infrastructure in Amazon EC2 At OpenX we recently completed a large-scale deployment of one of our server farms to Amazon EC2. Here are some lessons learned from that experience. Expect failures; what's more, embrace them Fully automate your infrastructure deployments Design your infrastructure so that it scales horizontally Establish clear measurable goals Be prepared to quickly identify and eliminate bottlenecks As you carefully watch various indicators of your systems' health, be prepared to.... Play wack-a-mole for a while, until things get stable

Two links to Berkun in one day! But there is a great axiom contained within: "Requirements is not Design."

The one book anyone working on requirements needs to read is Exploring Requirements by Gerald Weinberg. It points out most of the stupidity that goes on, explains avoidance tactics, and clearly expresses how requirements are part of the design process - that good problem solving techniques can quickly make your requirements documents better than ever.

Here’s a requirements list: Make a $5 car that goes 500 miles per hour, weighs 10 lbs, and is invisible. Those are very clear requirements. They’re also impossible.

"I don’t want an engineer working on something s/he thinks is stupid. How I can expect them to do good work on something they find stupid?"

1: Requirements is not Design. 2: Too Many Cooks. I like Berkun.

(1) Requirements may be impossible to implement. (2) Requirements writing is hard when no single person has the knowledge and authority.

At Hashrocket we use git both internally and in our Agile mentoring and training. Git gives us the flexibility to design a version control workflow that meets the needs of either a fully Agile team or a team that is transitioning towards an Agile process.

git config alias.datetag '!git tag `date "+%Y%m%d%H%M"`'

What's happened is that they haven't paid enough attention to the internal quality of their software. If you make that mistake you'll soon find your productivity dragged down because it's much harder to add new features than you'd like. You've taken on a crippling TechnicalDebt and your scrum has gone weak at the knees. (And if you've been in a real scrum, you'll know that's a Bad Thing.)

"XPers often joke, with some justification, that Scrum is just XP without the technical practices that make it work."

Individuals and Interactions are more valuable than Processes and Tools

Intrigued by microframeworks.

The trickiest problem I still need to solve is how to replace settings.py. A group of developers (including Adrian, Armin, Alex and myself) had an excellent brainstorming session at EuroDjangoCon about this. We realised that most of the stuff in settings.py can be recast as configuring services which Django makes available to the applications it is hosting.

I like the idea of a one-file microframework for websites. It is like SQLite instead of a fullfledged DBMS.

Simon Willison on his __djng microframework__ (which depends on Django). I share his hatred for settings.py and I'll be keeping an eye on this one.

Recommended TDD Tutorials

Recently, Dave Nicolette consolidated a list of recommended TDD tutorials from a discussion on the Extreme Programming group. Here is a sneak peak at the consolidated list with categorization for quickly getting started with Test Driven Development.

article on managing a startup and growing rapidly adapting to what users want

Many founders believe that early stage startups are endeavors of execution. The customer is known, the product is known, and all we have to do is act. Eric takes a different approach. He believes that many early stage startups are labors of learning. The customer is unknown, the product is unknown, and startups must be built to learn.

I don’t know. Sometimes I feel like some of these folks try too hard to soft-sell the excellent way presented by test-driven development (TDD) and pair programming (PP). A lot of alleged agilists make this argument: you see, by PP’ing and TDDing, you will greatly increase your quality, so even though your productivity goes down, you get the coolest glow by knowing in your heart of hearts that you are a good citizen of the coding world. I call bullshit. Not only is it not true that you can trade internal quality for more features, it’s actually the exact opposite case: the more productivity you seek, the higher you should raise your standard of internal quality. That’s right, that’s what GeePaw says. If you want more production, look first to raising your internal quality. So, how can such things be? 1. Because internal quality and external quality are not the same things. 2. Because the biggest single determinant in today’s production is the quality of yesterday’s product

Towards a Way of Excellence

Sometimes I feel like some of these folks try too hard to soft-sell the excellent way presented by test-driven development (TDD) and pair programming (PP). A lot of alleged agilists make this argument: you see, by PP’ing and TDDing, you will greatly increase your quality, so even though your productivity goes down, you get the coolest glow by knowing in your heart of hearts that you are a good citizen of the coding world. I call bullshit.

article on test-driven development

As a profession, we do a lot more software maintenance than we do greenfield development. And it’s at the maintenance end where TDD really pays off. I’m starting to see l

Zen is a simple, flexible, and cost-effective project management solution. Zen provides an web-based kanban board and a powerful messaging system to help you work more effectively with your team.

Agile web2.0 project mgm saas

Using Wikis in Project Planning

Elliotte Rusty Harold's perspective on six reasons that he believes pair programming succeeds: 1) Continuous code review, 2) Fewer blockages, 3) Masking distractions, 4) Guaranteed focus, 5) Multiple points of view, and 6) Reduced training cost and time.

Pair programming is like magic in more ways than one. It dramatically improves programmer productivity and reduces bug count, and yet it does so through a technique that’s completely counter-intuitive.

1. Continuous Code Review; 2. Fewer blockages; 3; Masking distractions; 4. Guaranteed focus; 5. Multiple points of view; 6.Reduced training cost and time

Invest in technical debts that may never come due.

In a startup, we should take full advantage of our options, even if they feel dirty or riddled with technical debt. Those moralizing feelings are not always reliable. In particular, try these three things: Invest in technical debts that may never come due. The biggest source of waste in new product development is building something that nobody wants. This is a sad outcome which we should work very hard to avoid

bpettichord thoughtful discussion of the pros & cons of technical debt. http://bit.ly/tDjuI #yam http://twitter.com/bpettichord/status/3111092688

Tom DeMarco, one of my heroes, is brave, and has adapted his earlier ideas

"I’m gradually coming to the conclusion that software engineering is an idea whose time has come and gone. I still believe it makes excellent sense to engineer software. But that isn’t exactly what software engineering has come to mean. "

Software Engineering...

DeMarco Reflects on 40 Years of Software Engineering Evolution

"99% Statement Coverage, 95% Branch Coverage, Goal: 100% branch coverage by Dec 2009, Striving for 100% test coverage has been [their] most effective method for finding bugs."

I didn't know gcc could do that. Cool.

@newsycombinator: "SQLite: A Lesson In Low-Defect Software (review/summary w/ working source) http://bit.ly/qcfy0" (from http://twitter.com/newsycombinator/status/3550078799)

Article about TDD methodology and practices.

a result if Professor Plum is changing some code

erview week rotating on actual production code, pairing with the same people they'll be working with if they're hired. Everyone gets a say in whether to hire a new recruit, and hesitance from a developer that actually paired with them means they do not get hired.

People often ask how can I justify two people when one will do. Will this not, just double my cost? To answer this question I think it is important to discuss what pair-programing is not.

At the simplest level pairing is helping people one-on-one, not through emails, let me show you behind a keyboard, instead of telling you in abstract. The amount of pairing and keeping each other on track depends on many things, but you will find that more you pair the more you will learn from each other and the better the code becomes.

Must show this to a few people.

I sacrificed my productivity to make Arthur productive in a single day. If I did not it would take Arthur weeks before he would figure out how to set everything up how things worked and enough courage to fix a bug. Yet that is exactly what most companies do. Think about the confidence Arthur had on day one working with us. He was up and ready and he fixed two bugs on day one.

We used this pretty effectively at Reevoo, and I'm sure I'll use it again.

technical debt

Analysis of when and why having bad / no tech design does / doesn't pay. Basically it's a metaphor for thinking about how and why we make tech design decisions.

What we actually know about software development, and why we believe it’s true.

Several useful statistics on what drives programmer performance and causes software projects to fail.

Agile projects aren't yet fully user-driven, but new research shows that developers are actually more bullish on key user experience issues than UX people themselves.

http://www.useit.com/alertbox/agile-methods.html Agile projects aren't yet fully user-driven, but new research shows that developers are actually more bullish on key user experience issues than UX people themselves. # Separate design and development, and have the user interface team progress one step ahead of the implementation team. That way, when it comes time to build something, it's already been designed and tested. (And yes, you can do both in a week or two by using paper prototypes and discount user testing.) # Maintain a coherent vision of the user interface architecture. Create the initial vision during a "sprint zero" period — before any implementation has started — and maintain it through annual (or semi-annual) design vision sprints. You can't just design individual features; they have to fit together into a coherent whole — a whole that must be designed as well. Bottom-up user interface design equals a confused total user experience (the Linux syndrome).

Distributing your UX personnel doesn't mean you have to abandon all the benefits of having a centralized, specialized group. Often, a matrix structure provides a good compromise, making UX professionals part of individual projects on a day-to-day basis, but still offering some company-wide coordination.

Nothing new, but fun to throw at the suits - "Clearly, Agile is considerably better than the old Waterfall method. Good riddance to that one. However, the professionals in our new study still felt that Iterative Design was marginally better than Agile"

Critical discussion about unit tests. It's important not to dismiss his core argument, that unit testing isn't the only testing you should be doing.

Don't feel bad, it's okay not to write unit tests. But Unit Tests Work For Me! First, are you sure you're really unit testing? Unit testing is all about testing unitsindependent pieces of logic. According to Michael Feathers, a real unit test cannot

argues that the process of requirements elicitation as project definition are secondary to the process of creation,testing and learning about nature.

What I cannot create I do not understand. --- Learn how to solve every problem that have been solved. --------- 1) The Space Shuttle Main Engine... many different kinds of flaws and difficulties have turned up. Because, unfortunately, it was built in the top-down manner, they are difficult to find and fix. 2) avionics system, which was done by a different group at NASA: The software is checked very carefully in a bottom-up fashion. First, each new line of code is checked, then sections of code or modules with special functions are verified.

アジャイルではない。チケットの粒度は一週間というのは大きく感じる。もちろん案件の規模間に左右されるのだろうけど。フリーのJIRA。エンタープライズにはもういくばくかの機能が足

Can get 10 user licenses of JIRA, Confluence or Bamboo for $10 each.

Our best tools are just $10 per product. Perfect for startups and small teams.

Jira - bug tracking Greenhopper - jira agile plugin confluence- wiki fisheye- version tracker

Start-up priced software from Atlassian - $10.

I'm on the fence about this. There's a lot of comparison of stuff that can't figure out how to make money (Facebook, Twitter) to stuff that needs to make money/simplify stuff/track money (CRM, accounting, etc.).

Tim Bray rant on the difference in user experience between Web consumer sites and Enterprise IT. "Obviously, the technology matters. This isn’t the place for details, but apparently the winning mix includes dynamic languages and Web frameworks and TDD and REST and Open Source and NoSQL [...] More important is the culture: iterative development, continuous refactoring, ubiquitous unit testing, starting small, gathering user experience before it seems reasonable. All of which, to be fair, I suppose had its roots in last decade’s Extreme and Agile movements. " His conclusion is: "Plan A: Don’t Build Systems [...as defended by ] Nicholas Carr: everything would be better if we could do IT the way we do electricity Plan B: Do it better: [but you cannot do it without adopting the agile ways of the web startup] that kind of thing simply cannot be built if you start with large formal specifications and fixed-price contracts and change-control procedures and so on."

The community of developers whose work you see on the Web, who probably don’t know what ADO or UML or JPA even stand for, deploy better systems at less cost in less time at lower risk than we see in the Enterprise.

pretty good about going with the web dev model. But then he says buy Oracle or SAP which I don't understand.

Study on REST and Enterprise Development

The games industry is rushing headlong to Agile development methodologies just now; it’s a great source of excitement for some, with conference sessions and magazine articles left, right and centre, and “evangelists” spreading the word. I’m sick of it. I can’t wait for the day when everyone realises how much of a fad-diet, religious-cult-inspired, money-making exercise it is for a group of consultants.

Agile doesn't mean no planning or allowing to reject any kind of "older" approarches, which are actually necessary to do the things.

All methodologies imply a prescribed approach, a single-minded, fixed set of processes that removes flexibility and rationality.

i wish i knew ;) - added by harper reed's google reader

great article on how and why to be a lean startup

Discussion of the principals of Lean Startups with many useful hints.

How I manage my most valuable asset – Time

Project Tracking Software - Less Projects http://bit.ly/6xugnj /via @lifehacker Congrats @lessallan @stevenbristol #simple

Project Task Management Application

e] I have many successful

this is a great article

"UCD can be too documentation-heavy, isolated and risky but Agile needs help with defining requirements and concept development. How can Agile and user centric principles work together?"

Some details about the mechanics of Five Whys

Annotated link http://www.diigo.com/bookmark/http%3A%2F%2Fstartuplessonslearned.blogspot.com%2F2009%2F07%2Fhow-to-conduct-five-whys-root-cause.html

Ironing out the little problems can make it so companies can avoid big disasters. At 24:14

pixar guy talks about their process

Very nice talk, lot's of agile aspects in this story about what went wrong at the succefull Pixar company

Fantastique vidéo de Ed Catmull sur les méthodologies de travail des films chez Pixar et sur les processus de Pixar en tant que compagnie.

how pixar builds high performance teams - codinghorror.com

Ed Catmull, Pixar: Keep Your Crises Small

Ironing out the little problems can make it so companies can avoid big disasters. Recorded: January 31, 2007Related Article: http://www.gsb.stanford.edu/ne...

The goal of continuous deployment is to help development teams drive waste out of their process by simultaneously reducing the batch size and increasing the tempo of their work.

the more he says it, the more it makes sense

In this presentation I give some examples of heavy weight methods and their implications on your project. Then I give a quick overview of Agile methods, the rationale behind it, its origin, its values and principles. I move on to

This is supposed to be an introductory presentation on Agile. In this presentation I give some examples of heavy weight methods and their implications on your project. Then I give a quick overview of Agile methods, the rationale behind it, its origin, its values and principles. I move on to describe that what I see happening today in the industry is really waterfall in the name of Agile. I give some reasons why this is happening and then I give some pointers to move away from this flawed thinking. Bottom line, Agile is not a Silver Bullet and don’t fall pray to marketing gimmicks. Question dogmatic claims. Adapt Agile to your needs and take baby steps.

Neben dem Film mein Favorit: Top 10 Code-Gerüche in Unternehmen

http://qconsf.com/sf2008/file?path=/qcon-sanfran-2008/slides//NealFord_10_Ways_to_Improve_Your_Code.pdf

An infoq presentation on 10 Ways to Improve Your Code...

Best practices and tips for Ruby RSpec

"After a year using RSpec, I’m happy to share “(My) RSpec Best Practices and Tips”. Let’s make your specs easier to maintain, less verbose, more structured and covering more cases!"

How to write more readable rspec tests

After a year using RSpec, I’m happy to share “(My) RSpec Best Practices and Tips”. Let’s make your specs easier to maintain, less verbose, more structured and covering more cases!

Lots of great anti-patterns. I especially like The Ugly Mirror

Jason Rudolph - Programmer, consultant, speaker, and author specializing in Ruby, Rails, Groovy, and Grails

worried

On Excessive Unit Testing and excessive design

From Podcast 38 - Joel on Software Wanted: Business Analyst (Agile) at redbox (Oakbrook Terrace, IL). See this and other great job listings on the jobs page. Here’s a brief conversation between Jeff and I which I transcribed from... はてなブックマーク - From Podcast 38 - Joel on Software はてなブックマークに追加 dann dann testing, good

Joel Spolsky blog

This post is a compilation of recommended Agile books by various Agilists. The recommendations try to cover the entire spectrum of process, people and technology related to Agile. The idea is to make the process of Agile adoption easier and fruitful.

"Essentially, it is in your best interest, as a developer, as a team, to encourage lots and lots of tests lower in the stacks shown here. It starts with comprehensive, checked in unit tests. It continues with having a strong, repeatable testing discipline (for which I recommend test automation)."

As a long-time automation-engineer/test-focused guy I've pondered the great existential question of "how much testing" is enough for awhile. More recently, I've started focusing on the cost of not testing a product.

a great pragmatic look at the value of unit tests in context of total time allocated to project, risk management, acceptance and functional testing. some common issues with unit testing illuminated. a great read.

Trust your programmers. They are not only engineers and craftsmen, but businessmen as well. They will know from experience what happens to client relationships when someone makes promises no one can keep.

Programming is a bit of mystery to people. It breeds a lot of distrust in an organization.

Actually the conclusion is 'Trust your programmers' just in case you were wondering

Code is invisible to everyone but the programmer. It’s like the magic that takes place behind the curtain. Any team has to just trust a programmer about what is and what isn’t doable. The best course of action, estimates, and progress updates are all subject to the programmer’s opinion. There’s plenty of Type A personalities that have trouble with this, but it goes deeper.

Agilists favor a open team room as it promotes lots of informal and deep communication between people on the team.

Team room is great but open space isn't, and you readit carefully you'll find the reason in the text: "(..) It isn't comparable to an open-plan office where everyone is doing something different".

Thoughts about how to create and organize a team room

common thing you find in agile projects is that the development team sits in a single open team room

Kostenloses Online-Buch für die Entwicklung mit Scrum

Scrum ist ein Framework für die Abwicklung von Projekten, das auf den Rahmengrundsätzen der agilen Softwareentwicklung aufbaut. »Scrum Kompakt« führt ein in die Grundlagen des Projektmanagements, bietet eine Einführung in Scrum und zeigt eine Variante, wie in Scrum Anforderungsanalyse mit hoher Qualität betrieben werden kann.

Scrum Kompakt bietet eine Einführung in Scrum und führt ein in die Grundlagen des Projektmanagements.