Lifestyle tools

by Paul Agapow
in programming

tagged programming, plone, opinion, rant, bioinformatics, c++, web-development

All I wanted to do was compile this code.

I'll stick with LaTeX, thank you very much. Which is poking yourself in the eye with a tool perfectly designed for eye-poking after you get the configuration right, but the tool can't be held by human hands so you also need a pneumatic dingus to poke the poker and a robot arm to hold the poker+dingus combination. But man, your eye is really beautifully poked.

(commenter on Metafilter)

I've wasted too much time over the years on software, libraries and systems that promised a lot but ended up consuming huge amounts of my time. After the first few times, I started to develop a wary eye for software that demanded I do nothing but tend to it, and tools that you couldn't just adopt and solve a problem with, but that seemed to take all your available time just to get your job done. This is especially problematic when you're a lone developer, in a small team or (like me) use software simply as a means to an end.

So here's some hard-won ideas about these "lifestyle" tools.

Note

I've put this piece in the 'programming' category, but it has a much wider occurence, from web development to data science and bioinformatics. Anywhere where there are tough problems, geeks and a lot of activity, you can expect lifestyle tools to pop up, promising to solve all your problems at the expense of all your time.

Ways to recognise lifestyle tools

The software

It hardly seems worth even having a bug system if the frequency of from-scratch rewrites always outstrips the pace of bug fixing ... But that's what happens when there is no incentive for people to do the parts of programming that aren't fun. Fixing bugs isn't fun.

(Jamie Zawinski)

  • NIH (not invented here): It reinvents the wheel by providing its own idiosyncratic solutions to problems that have been repeatedly solved elsewhere, i.e. its own build solution, templating language, logging or database interface.
  • Functionality is an ever-moving target: You can never get things to "just work". There's a niggling bug in the UI, the database connection only works with the version you got out of the repo, you had to write a birdnest of code to patch it but "it will be easier in the next version".
  • Pervasive and ongoing incompatibility without and within: "Oh yeah, that plugin doesn't work with version X. Get the new version from the repo / recompile all your tools and libraries / it'll be fixed in the next version ..."
  • Has over-engineered tools to solve problems that shouldn't exist: i.e. build tools to manage build tools, complex caching to solve inherent performance issues,
  • Ornate configuration files and schema: And they're almost always in XML.
  • Logging: for some reason, lifestyle tools show an obsession with logging, perhaps because the only way you'll work out what's going wrong is by crawling through a gigabyte of function calls. It may also be connected with their abysmal error-handling.
  • Bug reports go unfixed: Possibly because no one can reproduce your error and/or the setup is so complex anything could be happening.
  • Feature churn: that horribly complex and poorly documented class definition system? We're not bothering to fix it because the Brand New Hotness is in the repo and it's nearly ready for use but undocumented and it would be too hard to fix the old one ... (See Jamie Zawinski's CADT model.)
  • System complexity increases monotonically with time: Things never get better and simpler, because you're patching over fundamentally rotten foundations that few understand and developers can't stop themselves from adding another layer.

The documentation

That's one sure tip-off to the fact that you're being assaulted by an Architecture Astronaut: the incredible amount of bombast; the heroic, utopian grandiloquence; the boastfulness; the complete lack of reality. And people buy it! The business press goes wild! The hallmark of an architecture astronaut is that they don't solve an actual problem... they solve something that appears to be the template of a lot of problems. Or at least, they try.

(Joel Spolsky)

  • Is patchy: No one really understands it well enough, the task of documenting the whole system is too great and anyway, who has time after rewriting your configuration files for the second time this week.
  • Requests for clarification are answered with "Read the source": Because when I want to know how my microwave works, the best way is to dismantle it and read the engineering schematics.
  • The documentation has no clear entry-point: The complexity and interconnectivity of the product prevents any reasonable mile-high view. To do anything you have to know everything.
  • "You don't do it that way any more.": The effort of documenting system features - and the churn of features - means that documentation is polluted by references to old and outdated features.

The community

The community is massive ECHO chamber, where nothing gets out, seriously no one cares. Even worse, nothing gets in, we are on a desert island, isolated, alone at sea.

(Stevan Little)

  • Is large and (overly) active: Forums churn constantly with queries, because everyone has questions, the answers are not easy and it's often not clear what the correct answer is.
  • Reflexively and globally rejects any criticism: Raise even the smallest complaint about (say) arcane terminology, software instability or poor documentation and you can expect the pack to start baying. "What are you saying? There's nothing wrong with that! Why my grandmother managed to set up a server! With no instructions! In 10 minutes!" This may be something to do with human psychology and sunk costs.
  • Explanations are rife with jargon: "Override your XAML template interface with a monkey-patched factory-factory product, adding a new schema supermodel." This goes double if the new terms duplicate already existing ones, or use existing ones but in a different way.
  • You have nothing better to do: Users are expected to spend all day keeping up-to-date, knowing instantly about changes and new releases, reading all the mailing lists. Heaven forbid that you actually spend time doing something with the software.
  • Hyperbole and dismissal: Adherents will claim that the software is good for every situation and/or that if you can't get it to work "It's clearly not for you / It works here, it must be your problem".
  • 100% of analysts sitting at my desk use ...: The community clings to surveys and reports that cast the software in a good light. X had fewer security alerts last year than PHP! Yes, and it also had 1% of the users.
  • You suck!: A popular feature of the community are conference talks and blog posts that "demolish myths", by claiming that the problems don't exist, you're doing it wrong, how their product is secretly the innovative edge in it's domain that's inspiring all others, that if you're not prepared to help develop the product then GTFO, etc. etc. Adherents might also answer criticism by disingeniously wondering 'why are there are these haters' or arguing that complexity and unreliability are actually good things.
  • A closed system: Despite its (focused) expertise, the community is hermetic. You never meet anyone who is an enthusiast or expert in a lifestyle tool and something else. They will claim that no other system allows metaprogramming / is a more flexible CMS / has such unprecedented visualisation capabilities. Maybe this is because of their lack of knowledge, maybe they're just repeating talking points that have bounced around the community echo chamber without the benefit of verification.
  • Do more: The answer to any question is always to double-down: invest more time. Invest more effort. Use this new tool straight out of repo. Sift through the source. Then your code will work.

Where do lifestyle tools come from?

X Windows Is "customizable" ... And so is a molten blob of pig iron.

(Don Hopkins)

It's a good question. No one sets out to design one of these ugly, ungainly creatures, yet they keep cropping up.

One theory is that great developers (or developers that want to be seen as great or poor developers that want to mask their incompetence) wish to visibly demonstrate their "talent", so they develop mind-bendingly complex things that may not actually do the necessary job but are "so cool!"

Alternatively, I think that many examples share a common origin: at some point, they they were a decent solution to a particular problem when there were few alternatives. Their feature set was much smaller and they worked well enough. As time passed and they accumulated users and features, fundamental architecture changes become difficult. Small problems ossified into larger issues. Old hands in the community instinctively understand how everything works, even if that explanation is unbelievably complex. New hands spend so much time trying to get up to speed, they lose sight of what's happening outside the community. Eventually, everyone has invested so much time and effort in the system, that the idea of using something else ("throwing all that effort away") becomes anathema.

What happens to them?

On the upside, it seems to me that all lifestyle tools eventually perish. The size of and activity in the community will become unmanageable. Innovation grinds to a halt as few if any people understand what is going on. Someone will develop a simpler alternative. Projects based on the old tool will crash and burn - or simply fail to complete - while those based on alternatives will complete, the developers going on finish other projects. People begin to talk up the alternatives and slowly the old dogma dies down.

What do you do?

I suggest there is one key question to ask, to see if you're getting sucked into using a lifestyle tool:

Do you spend more time maintaining, learning about, debugging and managing the tool than doing anything with the problem domain you're supposed to be using the tool on?

Candidates

  • Maven: holy crap, it's just a build tool.
  • Eclipse: why do you need a tool to manage and update your IDE?
  • Mozilla: not just a browser but an entire cross-platform toolkit. Just read the instructions for setting-up the Mozilla development on your harddisk.
  • C++: Has arguably teetered on the edge of being a lifestyle tool, with features that were standardized before they were actually implemented, the overly verbose and cryptic template system, and puttering about minor syntax corrections. This may have been one of the reasons for the rise of Java ... which in turn has threatened to become a lifestyle tool. Maybe it's the large community that does it to a programming language.
  • Java frameworks and ORMs are legendary for their steep learning curve and eccentric behaviours (c.v Hibernate, J2EE, ), with huge amounts of energy being invested into mundane issues like how logging takes place.
  • Corba: Remember when Corba was going to solve all of our problems? Can you remember what it actually did or what those problems were?
  • Perl 6: it wasn't that many years ago that someone described their Perl-based webtool as "state of the art" and practitioners eagerly devoured the many lengthy missives on Perl's new features. Now the community is literally dying out as adherents age and retire.
  • Plone: guilty of most of the crimes above. This article was inspired by me noticing that mailing list activity had been dwindling steady for years, while several prominent community members had quietly slipped away. Seriously, the solution to theming is to invent a filtering proxy that sits in front of your website and rewrites all the HTML?.