Recap of the Schneide Dev Brunch 2017-10-08

brunch64-borderedLast sunday, we held another Schneide Dev Brunch, a regular brunch on the second sunday of every other (even) month, only that all attendees want to talk about software development and various other topics. This brunch was smaller, which enabled us to use the meeting table with some comfort. Sometimes, with many attendees and bad weather, this table can get a little bit crowded. As usual, the main theme was that if you bring a software-related topic along with your food, everyone has something to share. Because we were a small group, we discussed without an agenda. As usual, a lot of topics and chatter were exchanged. This recapitulation tries to highlight the main topics of the brunch, but cannot reiterate everything that was spoken. If you were there, you probably find this list inconclusive:

Area of training

We shared some stories about top-notch video game players and how they keep up with the demand to stay competitive. Similar stories can be told about every topic imagineable: What did the “king of the hill” do to rise to such levels? The answer is always: training. Excessive, brutal training. They are first in the gym and lock the door late at night because they are the last, too. The use every waking second for practice and repetition. They are obsessed with being “the best”. If you want to follow such a story in movie format, you might enjoy “Whiplash”, a movie about an aspiring expert drummer that also highlights the delicate trainer/trainee dynamics. If you are more interested in the strategies of obtaining mastery, the book “Mastery” by Robert Greene will give you a lot of insights.

With this background, we asked around what our area of training (not expertise, not mastery – just training) is. The answers varied wildly, from the obvious “programming” to “whisky” (as in whisky tasting and collecting whisky). It’s an interesting question: what goal are you actively pursuing at the moment?

Hacking challenges

Evolving from the first topic, we talked about coding challenges and “capture the flag” hacking contests. If you aren’t the grandmaster in the area of the contest, you’ll get the most out of it by following the participating teams and trying to understand their approaches.  The local security capture the flag team of the KIT is especially open with their approach, their failures and successes. You might want to check out their website.

One challenge included trying to break a whitebox encryption, which is an interesting topic in itself. Maybe somebody can read up on this topic and give a little presentation in the future. Another challenge seemed to lead to an elaborate buffer overflow attack, when in reality, it could be solved with a “simple” use-after-free attack.

An useful starting point for aspiring security hackers might be the CTF (capture the flag) field guide. There are also some online challenges for basic training purposes, like the cryptopals or the bandit wargame. Thanks Tobias for the links!

If you are more interested in playful challenges and don’t want to show up on somebody’s radar, programming/hacking games like TIS-100 are perfect for you. Our game night with TIS-100 is still in vivid memory.

Software Architecture training

There are a lot of programming contests and hacking challenges out there. But what about dry-run training for software architects? On a related scale, there are hundreds of training simulators for the foot soldier (called ego-shooters), but little games for the aspiring officer/general. The website armchair general lists a few and even has some contests, but they lack the depth of real experience. Similarly, the training for software architects will probably be clean-room exercises, when in reality, the customer needs, the team mood, the latest fad in technology and even the weather will influence the architecture just as well as textbook knowledge.

We couldn’t discuss this topic to its full potential, so it will re-appear on the agenda of the next Dev Brunch. And its open for discussion in the comments: What are good books and trainings for software architects?

Thomas pointed us to the Architectural Katas by Ted Neward. Perhaps we should schedule a Schneide Event to try them out?

Epilogue

As usual, the Dev Brunch contained a lot more chatter and talk than listed here. The number of attendees makes for an unique experience every time. We are looking forward to the next Dev Brunch at the Softwareschneiderei in December. We even have some topics still on the agenda (like a report about first-hand experiences with the programming language Rust). And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

Advertisements

The project jugglers

By Usien (Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia CommonsIn this blog post, I will shine some light on a feature of our company that is often met with disbelief: How five developers can work on twenty projects at once without being stressed. I try to use the metaphor of a juggler, though I know nothing about juggling other than it can be done. I cannot hold more than one object in the air at any given time and even that is an optimistic estimate. But I’ve seen jugglers keep six to eight objects flying with seemingly no effort. We do this with software development projects.

A layman theory of juggling

A good metaphor can be applied from start to finish. I’ve probably chosen a bad metaphor, but it gives the right initial impression: Every developer at our company leads several projects at once. He (or she) keeps the projects alive and in the “green zone”, the ratio of remaining budget, time and scope (read: work left to be done) that promises little to no trouble in the foreseeable future.

In order to juggle without visible effort, you probably need to practice a lot. You probably drop your objects a lot. You probably need to watch the objects fly in the beginning.

In our case, we needed a lot of practice to reach our level of confidence. We lead development projects for up to 17 years now. Each developer finishes between four and seven projects per year. That’s up to a hundred projects to gain experience from. But we couldn’t drop (read: fail) a lot of projects, because it directly hurt our bottom line. Just imagine you want to learn how to juggle, but all you got are expensive ming vases that you bought from your own money, without insurance. That’s how it feels to “experiment” with projects. So we play it safe and only accept projects we know we can handle. And we watch our projects fly, very very closely. In fact, we have a dedicated position, the “project manager”, with the one duty to periodically ask a bunch of questions to assure that the project is still in the green zone.

Draw the trajectory

Every object that you can juggle has its own characteristics of how it behaves once it is in the air. A good juggler can feel its trajectory and grab it in just the right moment before it would fall out of reach. The trick to keep a software development project in the green zone is the same: Get a hold on it before it ventures too far in an unfortunate direction, which is the natural tendency of all projects. The project lead has to periodically apply effort to keep the project afloat. But when is the right time to invest this effort? Spend it too soon and it has only minimal effect. Spend it too often and you’ll exhaust your power and your budget. But because every project has its own trajectory, too, and you can’t afford to let it slip, you should make the trajectory as visible as possible. You should draw it!

We’ve experimented with a lot of tools and visualizations. The one setup that works best for us is a low technology, high visibility approach. The project lead takes one half of a whiteboard and draws a classic burn-down chart or a variation (we often use a simple vertical progress bar). This chart is hand-drawn and rather crude, but big enough that everybody can see it. It is updated at least every time the project manager comes around to ask his or her questions. One of the questions actually is: “Is this chart up to date?”. The remaining budget of your project needs to be available at a glance, from across the hall. The project lead needs to “feel” this budget. And if it makes him or her nervous, it’s high time to determine the remaining scope and calendar time of the project once again.

In doing this positioning by triangulation on a regular schedule, the project lead draws the trajectory of the project for all three axes and can probably interpolate its future course. He can then apply effort to nudge (or yank, if things got worse fast) the project back on track.

Without the visibly drawn trajectory, your project lead is like a juggler in the dark, tossing unknown objects in the air and hoping that they’ll fall in place somehow.

Know your limit

As the complete noob to juggling that I am, I imagine that jugglers have a secret dress code like martial artists (watch their belts!), where other jugglers can read how many objects they can hold up at once. Something like buttons on the vest or the length of a scarf. So the beginning 3-objects juggler bows in awe to the master 12-juggler, who himself is star-struck by the mighty 18-juggler that happens to attend the same meet-up.

In our company, this “dress code” would be based on the number of projects you are leading. And just like with the jugglers, it is important that you know your current limit. There is no use in over-extending yourself, if you accidentally let one project slip, the impact is big enough that you’ll fail your other projects, too. Just like the juggler loses his or her rhythm, you’ll lose your “flow”.

The most important part of juggling many projects is that you always juggle one less than you are capable of. You need reaction time if one of them topples over. A good juggler can “rescue” the situation with subtle speedup or extra movements because the delay between necessary actions allows for it. A good project lead has emergency reserves to spend without compromising other projects.

There is nothing wrong to start with two projects and add more later on when you are more confident. But don’t start with only one. You can only form habits of resource sharing if you share from the beginning. Even I can pose as a competent 1-juggler, but the lowest bar to juggling has to be two objects.

Box your time

Again, I know nothing about juggling. But from a mathematical viewpoint, juggling is “just” an exercise in timeboxing. If you have four objects in the air, in an arc that requires one second to go around, you’ll be able to spend a quarter second (250 ms) of attention to each object on each rotation. The master 12-juggler from above can only afford 1/12th or 80 milliseconds for each object. If he takes longer for one object, the next one will suffer. If he has no time reserves, a jam will build up and ultimately break the routine.

So, as a project lead, you need to apply timeboxes on all of your projects. They don’t need to be of equal size, but small enough that you can multiplex between your projects fast and often enough. A time box is a fixed-size amount of time that you allot to a specific task. The juggler uses a time box to put the next falling object flying back up. In our lunch break, we allot 60 minutes to food, beverages and some amount of walking. And if the process of eating takes longer than usual (I’m a chronic slow-eater, I can’t help it)? Then I have to go back partially hungry because I’ll end my lunch break on time. That’s the most important characteristic of a time box: You either succeed “in time” or interrupt or even cancel the task. The juggler will drop the one iffy object instead of risking a complete breakdown of the arc. You need to let your problematic task go (for the moment) instead of spreading the problem onto your other projects, too.

We’ve found that the amount of “one workday” is the most natural and easiest to manage time box. So we try our best to partition our week in the granularity of days and not our days in the granularity of hours. One aspect that helps tremendously is to have different physical locations for different projects. So you can be physically present “in the project” or “too far away at the moment” from the project. You plan your work week in locations as much as you plan it in project time boxes. The correlation of workdays, locations and projects is so strong that it doesn’t even seem to be timeboxing or project multiplexing. You just happen to be in the right place to work on project X for today. This is how you can juggle up to five projects without having to compromise all that much (provided you have a five-day work week).

If you can’t physically relocate your work, at least try to have a fixed schedule for your projects, like the “project A monday” or the “project X friday”. This might also mean to postpone emerging issues with project X until next friday. You need to build up skills to negotiate these delays with your customers. If your customers can dictate your schedule, you’ll get torn to shreds in no time. It’s friday or no day for issues on project X – at least as a good start for heavy bargaining. But that’s a whole topic for another blog post. Please leave a comment if you are interested to hear more about it.

Stuff your box

The “one workday” time box has a strong implication: Every little thing you do for a project takes one day. That doesn’t mean you should work for five minutes on project A, completing the task, and then stare into the air and twiddle your thumbs. It means that you should accumulate enough tasks for project A that you can spend the better half of the day on the known tasks and the remaining time on the unknown problems that arise on the way. In the evening, you should be able to finish your work for project A with a feeling of closure. You can put project A aside until next week (or whenever your next cycle is). You can concentrate on project B tomorrow and project C the next day. Both projects didn’t bother you today (well, perhaps a bit, but you only acknowledged some e-mails and deferred any real thoughts on it until you enter their timebox).

Perpetual closure

The feeling of closure at the end of a successful work day is the most important thing that keeps you composed. You’ve done your thinking for project A this week and will think of project B tomorrow. But now, you can rest.

This must be the feeling that the juggler experiences with each object that goes up again. It is out of sight and only needs attention after it has nearly completed its arc again. And now for the next object, one at a time…

 

On the usefulness of volatile memory

I have a strange hobby: Raising ants as pets. Not literally pets, because I don’t give them names and they probably don’t know I even exist, but as tended animals in a controlled and restricted environment, the formicarium. This doesn’t sound too strange if you know three additional things about me: I was fascinated by ants since kindergarten, I play dwarf fortress for fun and I’m interested in computers.

My main colony is a lasius niger nest with about 1500 individuals. Lasius niger or black garden ant is a very common and easy to raise ant that is small enough to not require too much space but big enough to be observed and tracked.

I could tell you hours and hours of ant facts, but let’s concentrate on the topic of this blog post: An ant colony can be perceived as one big organism. Each individual ant has a clear role in the hierarchy:

  • The queen ant is the sole egg layer. In an established colony, she won’t do anything else. She will be fed, cleaned and protected by her workers. She was born a queen and cannot be replaced. If she dies, the whole colony will slowly fade away because nobody produces new ants anymore. Worker ants don’t know if they still have a queen and don’t care anyway. If a queen dies, she will be fed to the remaining larvas as soon as her scent disappears.
  • The male ants are born, fed and kicked out of the nest, preferably when the young queens fly out to find a mate. They won’t do anything else and have a lifespan of days.
  • The worker ants are all sterile females and do all the work. Literally all the work. They are born workers and spend every moment of their lives contributing to the hive or just sitting around scrounging food. Other workers are too busy to judge them.

But keep in mind that ants only have short-range communication means (scent, antennae drumming and ground vibrations), so no single ant has complete overview about the situation. It wouldn’t have the brains to process that information anyway. Ants have limited capability to remember things but no long-time memory. It isn’t necessary for the single worker ant to store any information. The information is stored in the hive – literally.

If you abstract some details away, you can also perceive and describe an ant colony as a computer or at least as a massive parallel problem solver. The algorithms are ingrained in the worker ants and are executed ruthlessly. The problems are food, water, enemies, cleaning and nurture. If the environment (the problem space) is suitable for the algorithms, the colony will thrive. Else the colony will ultimately fail. An ant colony at the side of a busy road will lose many workers in sudden “enemy” attacks, while a colony in the middle of a meadow might find less insects killed by cars. Not a single ant in each colony is aware of these relations. But they have found a way to share information: They mark their position (and therefore their way) by special scent fluid. They use their scents to label the environment for other ants.

If you ever encountered an avid user of printed sticker labels, this is what ants do, too: They put a sticker label on everything they come in contact with. If you seem to be food, they label you as food and rush back to the hive, laying out a “food in this direction” lane. If you seem dead and inedible, they mark you as garbage and them or some other ant will pick you up and follow the garbage line (finding the garbage line often requires them to return near their hive so it seems they mistook the garbage for food). The garbage area is marked with its own scent, preferably at a cliff. My ants love to throw things down a cliff! If you didn’t get the relationship to dwarf fortress yet, it should be clear by now. There are countless more examples, but you get the idea. The environment is not only an area, it is the long-term memory of the colony. The colony’s “memory brain” is scent on dirt, stones and sticks. The ant colony has outsourced its complete knowledge about the surrounding into the surrounding itself.

The ants have invented something I would call an “inverse cloud”. The cloud in IT is a concept of data storage that exists mostly independent of physical location and provides access to that data from virtually anywhere. The ants’ inverse cloud is a concept of data storage that is tightly coupled to a physical location and provides access to that data only if you are in the immediate vincinity. If you remove a physical data storage part in the IT cloud, it gets replaced by other parts that contain mirrored copies of the data. The cloud never forgets. If you remove a physical data storage part in the ants’ inverse cloud, it is forgotten immediately. Ants accept their environment as it is right now and never look back.

Now think about what happens when it rains and all the scents are washed away.

After every rain, the ant colony enters a “new level”, a fresh environment to be discovered and labeled. They probably never grow old to rediscover the same cliff again and again. This is what happens to a computer when the power is lost. It loses its working memory. But keep in mind that the colony retains some memories: the hive is underground and often rain-proof by overlaying stones or plants. So the colony remembers that it is currently in the hive, because it always smells like hive. The colony remembers the queen chamber because it always smells like queen. The colony remembers the brood chambers because they always smell like teen spirit (SCNR).

In my formicarium, it never rains. The ants get enough water by drinking troughs, but the marked lanes are never erased. This leads to all sorts of silly situations that the ants don’t even recognize as such. For example, a strong “food here” scent lane exists long after the food is gone. So a lot of enthusiastic workers run around searching in the target area. And remember that the hive smell never fades? My ants have assimilated area after area as “hive” after enough ants have marked it. So now they react excessively to disturbances because they think they are defending the hive – and therefore the queen! – but are ant miles away from the colony. Even better, a lot of young ants that usually never leave the hive until they are older wander out into the open (“it’s still the hive, just less dark”) and panic as soon as they encounter something that shouldn’t be in a hive. The panic spreads by, you’ve guessed it already, alarm scent and soon hundreds of battle-ready ants are running around frantically without any one of them knowing why.

So, to speak in computer terms, the main memory never gets erased and the caches never flushed. A little error can spread like a wildfire (like the panic example above) and cause disadvantages like energy consumption without any real gain (no enemy to defend against) or a lot of delicate ants sitting around in plain sight of their predators. The whole system is fragile and erratic and probably wouldn’t survive in the wild. A good measure of rain would remove the odd memories and probably ease the ants because their hive, the area that needs to be defended at all costs, would shrink again.

I’ve raised neurotic ants in need of a cold shower.

How does this give us any insight into modern computing? Well, my train of thoughts is this: If modern systems are unable to forget, because memory is cheap and permanent, we might be prone to design software that acts neurotic and hyped up. The ability to forget, to really don’t remember at all, might be crucial in designing resilient parallel systems. There is the cost of losing valueable information, but the benefit of losing all results of errors seems to match it. So volatile memory might be a nuisance for us programmers, but it also provides a “blank slate” every time the system starts and is the reason for the most important question in IT: “Have you tried turning it off and on again?”

Systems that rely on “place oriented programming”  seem to have the need of regular reset phases where the working memory is cleared and the system goes into the next cycle fresh and rested. We might even call it sleep. And in case you wonder: The sleep of ants is an ongoing topic for research.

Disclaimer: I know that not all ants are as dumb as lasius niger. Some ants even teach each other facts about their environment. The wikipedia article mentions some wonderful examples. I had ant colonies with more complex ants and they were wonderful. But right now, as I’m typing this, there is a lasius niger worker that heaves a wasp husk part to the top corner of the formicarium, throws it down (did I mention they love throwing things?) and runs back down to heave it up again, probably because the corner is somehow marked as a garbage dump zone. It has repeated this process at least half a dozen times now. This is how a biological infinite loop looks like. Some ants even parallelize such a loop to exhaustion:

Learning about Class Literals after twenty years of Java

I’ve programmed in Java nearly every day for twenty years now. At the beginning of my computer science studies, I was introduced to Java 1.0.x and have since accompanied every version of Java. Our professor made us buy the Java Language Specification on paper (it was quite a large book even back then) and I occassionally read it like you would read an encyclopedia – wading through a lot of already known facts just to discover something surprising and interesting, no matter how small.

With the end of my studies came the end of random research in old books – new books had to be read and understood. It was no longer efficient enough to randomly spend time with something, everything needed to have a clear goal, an outcome that improved my current position. This made me very efficient and quite effective, but I only uncover surprising facts and finds now if work forces me to go there.

An odd customer request

Recently, my work required me to re-visit an old acquaintance in the Java API that I’ve never grew fond of: The Runtime.exec() method. One of my customer had an recurring hardware problem that could only be solved by rebooting the machine. My software could already detect the symptoms of the problem and notify the operator, but the next logical step was to enable the software to perform the reboot on its own. The customer was very aware of the risks for such a functionality – I consider it a “sabotage feature”, but asked for it anyway. Because the software is written in Java, the reboot should be written in Java, too. And because the target machines are exclusively running on Windows, it was a viable option to implement the feature for that specific platform. Which brings me to Runtime.exec().

A simple solution for the reboot functionality in Java on Windows looks like this:


Runtime.exec("shutdown /r");

With this solution, the user is informed of the imminent reboot and has some time to make a decision. In my case, the reboot needed to be performed as fast as possible to minimize the loss of sensor data. So the reboot command needs to be expanded by a parameter:


Runtime.exec("shutdown /r /t 0");

And this is when the command stops working and politely tells you that you messed up the command line by printing the usage information. Which, of course, you can only see if you drain the output stream of the Process instance that performs the command in the background:


final Process process = Runtime.exec("shutdown /r /t 0");
try (final Scanner output = new Scanner(process.getInputStream())) {
    while (output.hasNextLine()) {
        System.out.println(output.nextLine());
    }
}

The output draining is good practice anyway, because the Process will just stop once the buffer is filled up. Which you will never see in development, but definitely in production – in the middle of the night on a weekend when you are on vacaction.

Modern thinking

In Java 5 and improved in Java 7, the Runtime.exec() method got less attractive by the introduction of the ProcessBuilder, a class that improves the experience of creating a correct command line and a lot of other things. So let’s switch to the ProcessBuilder:


final ProcessBuilder builder = new ProcessBuilder(
        "shutdown",
        "/r",
        "/t 0");
final Process process = builder.start();

Didn’t change a thing. The shutdown command still informs us that we don’t have our command line under control. And that’s true: The whole API is notorious of not telling me what is really going on in the background. The ProcessBuilder could be nice and offer a method that returns a String as it is issued to the operating system, but all we got is the ProcessBuilder.command() method that gives us the same command line parts we gave it. The mystery begins with our call of ProcessBuilder.start(), because it delegates to a class called ProcessImpl, and more specific to the static method ProcessImpl.start().

In this method, Java calls the private constructor of ProcessImpl, that performs a lot of black magic on our command line parts and ultimately disappears in a native method called create() with the actual command line (called cmdstr) as the first parameter. That’s the information I was looking for! In newer Java versions (starting with Java 7), the cmdstr is built in a private static method of ProcessImpl: ProcessImpl.createCommandLine(). If I could write a test program that calls this method directly, I would be able to see the actual command line by myself.

Disclaimer: I’m not an advocate of light-hearted use of the reflection API of Java. But for one-off programs, it’s a very powerful tool that gets the job done.

So let’s write the code to retrieve our actual command line directly from the ProcessImpl.createCommandLine() method:


public static void main(final String[] args) throws Exception {
    final String[] cmd = {
            "shutdown.exe",
            "/r",
            "/t 0",
    };
    final String executablePath = new File(cmd[0]).getPath();

    final Class<?> impl = ClassLoader.getSystemClassLoader().loadClass("java.lang.ProcessImpl");
    final Method myMethod = impl.getDeclaredMethod(
            "createCommandLine",
            new Class[] {
                    ????, // <-- Damn, I don't have any clue what should go here.
                    String.class,
                    String[].class
            });
    myMethod.setAccessible(true);

    final Object result = myMethod.invoke(
            null,
            2,
            executablePath,
            cmd);
    System.out.println(result);
}

The discovery

You probably noticed the “????” entry in the code above. That’s the discovery part I want to tell you about. This is when I met Class Literals in the Java Language Specification in chapter 15.8.2 (go and look it up!). The signature of the createCommandLine method is:


private static String createCommandLine(
        int verificationType,
        final String executablePath,
        final String cmd[])

Note: I didn’t remove the final keyword of verificationType, it isn’t there in the original code for unknown reasons.
When I wrote the reflection code above, it occurred to me that I had never attempted to lookup a method that contains a primitive parameter – the int in this case. I didn’t think much about it and went with Integer.class, but that didn’t work. And then, my discovery started:


final Method myMethod = impl.getDeclaredMethod(
        "createCommandLine",
        new Class[] {
                int.class, // <-- Look what I can do!
                String.class,
                String[].class
        });

As stated in the Java Language Specification, every primitive type of Java conceptionally “has” a public static field named “class” that contains the Class object for this primitive. We can even type void.class and gain access to the Class object of void. This is clearly written in the language specification and required knowledge for every earnest usage of Java’s reflection capabilities, but I somehow evaded it for twenty years.

I love when moments like this happen. I always feel dumb and enlightened at the same time and assume that everybody around me knew this fact for years, it is just me that didn’t get the memo.

The solution

Oh, and before I forget it, the solution to the reboot command not working is the odd way in which Java adds quote characters to the command line. The output above is:


shutdown /r "/t 0"

The extra quotes around /t 0 make the shutdown command reject all parameters and print the usage text instead. A working, if not necessarily intuitive solution is to separate the /t parameter and its value in order to never have spaces in the parameters – this is what provokes Java to try to help you by quoting the whole parameter (and is considered a feature rather than a bug):


final String[] cmd = {
        "shutdown",
        "/r",
        "/t",
        "0",
};

This results in the command line I wanted from the start:


shutdown /r /t 0

And reboots the computer instantaneous. Try it!

Your story?

What’s your “damn, I must’ve missed the memo” moment in the programming language you know best?

The personal economics of programming languages

Recently, one of my students asked a good question about what programming languages I would recommend learning. His ideal language would be “syntactically ugly, but giving insights that are universal to programming”. My first reaction was to answer that he has just described Perl, but that was too easy of an answer. So I tried to define the basics about programming languages, starting with the personal economics.

Economics of programming languages

An organization that wants to produce a piece of software needs to answer a lot of questions like “what programming language will be best suited for the task?”. Often, these questions get diluted and rather sound like “what programming language should we stipulate for all our projects, now and forever?”. That’s when politics and economics overlap and intermingle. We can leave this problem for the organizations to solve themselves. But if we scale the question down to an individual programmer – you, what influences are there to find an answer to the question “what programming languages should I learn?”.

I try to answer with the concept of utility: Learn those languages that, over a reasonable time, yield the most “utility”. There are at least two types of utility in our profession: money and joy. You can learn a programming language because your job requires it (money) or because you are curious and/or dig its particularities (joy). Most of the time, a specific programming language contains a mixture of both utilities for you. How you rate those utilities is up to you and probably varies from situation to situation. If you start a private fun project, picking the boring mainstream language from work might get things get done faster, but when would you want the fun to be over sooner?

Let me give two extreme examples for this concept:

  • If you start to learn COBOL now, chances are high that you will achieve two things: You will be disgusted by the language and the existing codebase, but delighted by the salary and job security. COBOL is a high money-utility programming language. It ranks low in any survey or statistics about programming languages, but is widely used in big business today and tomorrow. You might refer to https://blog.hackerrank.com/the-inevitable-return-of-cobol/ for more information.
  • If you start to learn Esterel now, you might experience two things over time: an epiphany about how flawed our concept of time is in most programming languages and an existential crisis because your brain isn’t capable to wrap itself around most sourcecode. Whatever comes first will define your learning success. There are virtually no jobs that require Esterel (even if some might benefit from it) and you can only program and build so many bicycle computers in your spare time (this is a typical introduction project to Esterel). Esterel is a pure joy-utility programming language. You can claim to be proficient in synchronous programming afterwards, but nobody will know what that even is.

A third type of utility

But I think that there might be a third type of utility for personal learning choices based on economics: The stirrup iron utility. Knowledge of some programming languages isn’t useful from a money-driven viewpoint and may lack enjoyability, but it serves as a door-opener to more enjoyable or sellable languages. It serves as an interim utility because it doesn’t have value in itself, but serves as a multiplier for either the money or joy utility. To rate the value of this utility to your career, you need to be clear about your career goals, especially your anticipated skill portfolio.

Skill portfolio shapes

Modern recruitment differentiates between several skill portfolio shapes, most noteably the “I” and “T” shape:

  • Programmers with “I”-shaped skill portfolios are experts in one specific field of programming. They might, for example, be the best C# programmer you’ve ever met. But they flop around like a fish out of water once they need to use another programming language. They will choose their familiar tools for every problem that needs to be solved and will solve it fast if possible or
  • Programmers with “T”-shaped skill portfolios have knowledge across all fields of programming, albeit limited, and drilled down into one field specifically. Why they chose to master their field can mostly be explained with the money or joy utility. They probably gained their broad knowledge base by using stirrup irons.

If you happen to know what’s expected from you until your retirement (let’s say you chose to program in COBOL), the “I”-shape is a viable and efficient strategy to manage your skill portfolio. There is nothing wrong with this approach (as long as it works).

If you have a hunch that you don’t have the capability to invest in broad knowledge, the “I”-shaped skill portfolio is your logical choice. It takes a lot to be able to come to a self-assessment that shows your limitations. It’s a good thing to know your limits and build a career within them. A lot of programmers don’t know their limits and burn out, because not meeting the requirements produce a lot of stress (on both sides). Better be yourself than over-promise and under-deliver constantly.

The “T”-shape means that you need to invest your time wisely. And we are not talking “work time” only, but “life time”, because you’ll probably need to spend your spare time working on your portfolio, too. Becoming a “jack of all trades” programmer is an endeavour of at least ten years without any possibility to shortcut. You need to select your jobs in accordance to your learning strategy and always be receptive to opportunities. You need to improve your learning abilities. You need to do so much at once that I suggest you start by watching Cory House’s talk about “Becoming an Outlier”. He’s spot on with so many things.

Stirrup iron programming languages

There are some programming languages that can be seen as the archetypes of a whole class of languages. Most knowledge of these archetypes can be directly applied or transfered to each language in the class. It’s the language’s concepts that are the real benefit. If you understand the synchronous programming aspect in Esterel, you’ll recognize it straight away in languages like LabView or SIGNAL. It may even just be a part of the other language (like in many multi-paradigm programming languages), but it will be familiar to you.

So what are some stirrup iron languages?

That’s a tough question and I want to place it out there. Can you drop a comment and name the programming language that had the most peculiar influence on your knowledge? I would like to refer to the book Seven Languages In Seven Weeks from the Pragmatic Bookshelf. It covers Ruby, Io, Prolog, Scala, Erlang, Clojure and Haskell. Do you agree with that selection? I would like to hear from you.

There are some ideas about this topic already: The talk “The Future of Programming” from Bret Victor (if you don’t know this guy already, please watch his legendary “Inventing on Principle” too). Richard Astbury presents three “new” hot programming languages (with matching outfits) in his talk “The State of the Art”. And Robert C. Martin is sure to have found “The Last Programming Language”.

One thing is sure: We should train the next generations of programmers in those stirrup iron languages, so they can quickly grasp the language flavour of the year. This is mostly done already, of course, but the students inevitably complain about the “weird” choices. So we need to explain upfront the economics of programming languages.

And, in a lighter tone at the end, there is always the ongoing competition for the worst programming language ever.

Eliminate the Water Carrier

Some years ago, an old lady with more than hundred years of life experience im America was asked which technology changed her life the most. She didn’t hesitate to answer: running water. The ability to open the tap and have instant access to fresh water was the single most important technology in her life, even before electricity and all the household appliances it enables. Without running water, every household is forced to employ or pay a worker that does nothing else but to carry water from the source to the sink.

In today’s physical world, with physical goods, there is still a profession that relies on a specific aspect of physical objects: They won’t move from A to B without a carrier. The whole field of logistics and transportation would be obsolete in the instant that physical goods learn to move themselves. The water carrier lives on, in the form of a cardboard or palette carrier.

The three basic goods of IT are software, data and information. They all share a common trait: They can move without a human carrier. In the old days before the internet, software was distributed on physical objects like floppy disks (think of oddly shaped usb sticks) or CDs later. With the ubiquitous access to running data (often called the internet and mobile computing), we can draw our software straight from the tap. (And yes, I like the metaphor of the modem as an “information tap”). As the data throughput of our internet connections grew, it became feasible to move large amounts of data into “the cloud”. The paper boy that brings the newspaper early every morning is replaced by a virtual newspaper that updates every few seconds. The profession of a data carrier didn’t exist outside of very delicate data movements. And even them got replaced by strong cryptography.

Even information and knowledge, a classic carrier-bound good, is slowly replaced by books and pre-recorded online courses. The “wise man” (or woman) still exists, but his range was extended from his immediate geographical surrounding and his arbirtrary placement on the timeline to the whole world and all times after his publication. We don’t need to be physically present to attend a course anymore and we don’t need to synchronize our schedule with the lecturer. Knowledge and information is free to roam the planet.

With all this said and known, why are there still jobs and activities that resemble nothing more than the water carrier of our information age? Let me reiterate once more what a water carrier does: He takes something from position A and moves it to position B. In the ideal case, everything he picked up at A is delivered at B, in full and unchanged. We don’t want the carrier to lose part of the water underway and we surely don’t want him to tamper with our water.

As soon as you add something valueable to the payload (you augment it) while you carry it from A to B, you aren’t a water carrier anymore, you can be described in terms of your augmentation. But what if you add nothing? If you deliver the payload in the same condition as you picked it up? Then you are a water carrier. You don’t have a justification for your work in IT. Or you have one that I can’t see right now, then I’m eager to hear from you! Please leave a comment.

There is a classic movie that describes life and work in IT perfectly: Office Space. If you haven’t seen it yet, please put it on your watch list. I’m sure you can even draw it from your information tap. In the movie, a company with a generic IT name needs to “consolidate their staff” (as in lose some slackers). They hire some consultants that interview the whole crew. Each interview is hilarious in itself, but one is funny, tragic and suitable for our topic at hand, the water carrier:

The problem with Tom Smykowski (the guy trying to defend his job) is, that he’s probably better with people than most developers, but he still cannot sell his augmentations to the two consultants. They try to tie him down to a physical good that must be carried, but even Tom has to admit that somebody else covers the physical level. So he tries to sell his “good influence” on the process as the augmentation, but the consultants are too ignorant to recognize it. Needless to say, Tom loses his job.

Every time you just relay information without transforming it (like appending additional information or condensing it to its essence), you just carry water. Improve your environment by bypassing yourself. If you take yourself out of the communication queue, you will save time and effort and nobody has a disadvantage. You should only be part of a communication or work queue if you can augment the thing being passed through the queue. If you can’t specify your augmentation, perhaps somebody else behind you in the queue can give you hints about it. I would argue that being able to pinpoint one’s contribution to the result is the most important part of every workplace description. If you know your contribution, you can improve it. Otherwise, you may be carrying water without even knowing it.

Eliminate the middlemen in your work queues to improve efficiency. But be sure to keep anybody who contributes to the result. So, eliminate the water carriers.

Recap of the Schneide Dev Brunch 2017-04-09

brunch64-borderedLast sunday, we held another Schneide Dev Brunch, a regular brunch on the second sunday of every other (even) month, only that all attendees want to talk about software development and various other topics. This brunch was well-attended and opened the sunroof season for us. We even had to take turns on the sunny places because we didn’t want to catch a sunburn in April. As usual, the main theme was that if you bring a software-related topic along with your food, everyone has something to share. Because we were very invested in our topics, we established an agenda for the event. As usual, a lot of topics and chatter were exchanged. This recapitulation tries to highlight the main topics of the brunch, but cannot reiterate everything that was spoken. If you were there, you probably find this list inconclusive:

Online courses

Our first topic was an report on an ongoing online course, a so-called MOOC (Massive Open Online Course) on the topic “Software Design for Non-Designers”. It aims at bringing basic knowledge of UX and UI design to programmers, who frequently lack even the most fundamental principles of design (other than code design and even that is open for discussion). A great advantage of these MOOCs is that you can minimize your brutto time investment and therefor maximize your netto yield. You are not bound to a certain place, free from specific times (other than the interaction with other participants) and yet free to engage in a community of peers. The question that remains is how valueable the certificate will be. But the initial expectations are met: The specific course is very practical and requires moderate effort in reasonable periods.

One crucial aspect is the professionality of the presenting lecturer. In this MOOC, there are talk-oriented presenters and then there is Scott Klemmer. His lectures stand out because he writes on an invisible wall before him. The camera looks through the wall. What seems like nice CGI turns out to be a real glass pane. Mr. Klemmer puts down his note in mirror writing! Once you realize that, you cannot help it but be in awe.

There are a lot of MOOCs nowadays. Other courses that got mentioned cover the topic of machine learning https://www.coursera.org/learn/machine-learning and Getting Started with Redux (a famous Javascript framework) by Dan Abramov on Egghead: https://egghead.io/courses/getting-started-with-redux. Some courses even take place on Youtube, if you manage to avoid the comment sections, like the talks from Geoffrey Hinton about neuronal networks and machine learning. Mr. Hinton is part of the Google Brain team.

The critical part of each MOOC is the final examination. Some courses require online or even real-time tests, some online provide certificates for test results in a certain timespan. Usually, the training assignments are peer reviewed by other course participants.

We will probably see this type of knowledge transfer more often in the future.

Interesting websites

While we talked about a lot of topics at once, some websites and projects got mentioned. I include them here without full coverage of the topics that led to it:

  • jsfiddle: A website that provides a quick sketchboard for web technologies like Javascript, HTML and CSS. It’s like a repl for the web.
  • regex101: A website that provides a quick sketchboard (and debugger) for regular expressions in different languages. It’s like an online IDE for regular expressions.
  • codefights: A website that puts you in the fighting pit for developers. Prove your programming skills against competition all around the globe!
  • vimgolf: A website that lets you prove your proficiency in the only text editor that counts: vim. Every keystroke counts and a mouse cannot be found!

Some of these websites might be a lot more fun in a team, except the regex one. Don’t use regular expressions in a team project! It’s a violation of the sane developer’s rules of engagement.

Workplace conflicts

One participant reported about his latest insights in conflict management during work. He applied the concepts of warfare and the four steps of complex tasks to recent disputes and had tremenduous results. Even the introduction chapter of the Strategies of War book was enough to install new notions and terms into his planning and acting. He was astounded by the positive effects of his new portfolio.

The new terminology seems to be the essential part. European (or even western) adults don’t learn the terminology of conflict and therefore cannot process disputes on a rational level, only with emotions. You cannot plan or communicate with emotions, so you cannot plan your conflict behaviour. As soon as you have the language to describe the things you perceive, you can analyze them, reflect on them and plan for them. Making a solid plan (other than “go in and win somehow”) is the best preparation for an upcoming conflict. Words shape our world. I’ve seldomly seen it clearer than in this report.

Just for starters, there is a difference between a “friend” and an “ally”.

Project documentation

An open question to all participants was our handling of documentation efforts in a project, be it for the user, customer or following developer. We discussed it with this open scope and came up with some tools that I can repeat here:

  • The arc42 software architecture template can help to shape the documentation effort for future developers or current developers if they aren’t included in the architecture effort.
  • The user manual is often written in TEX. Developers are used to the tool by constant exposition during their academic studies.
  • One idea was to generate the requirements for the developers from the user manual, as in “user manual first” or “user manual driven development”.
  • The good old Markdown syntax is useable but has its limits in top-notch aesthetics.
  • We see some potential in ASCIIDoc, but it needs to improve further to play in the same league as other tools.
  • Several participants have tried to automate the process of taking screenshots of the software for usage in various documents. If you want to try this, be warned! There are many detail problems that need to be solved before your solution will be fully automatic and reliable. A good starting point for thoughts is the “handbook data set” that can reproduce the same screenshot content (like entries in lists, etc.) in a different software version.

In the outskirt area of this discussion, the worthwhile talk “Stop Refactoring!” by Nat Pryce was mentioned. He presents an interesting take on the old question of “good enough”.

Epilogue

As usual, the Dev Brunch contained a lot more chatter and talk than listed here. The number of attendees makes for an unique experience every time. We are looking forward to the next Dev Brunch at the Softwareschneiderei in June. We even have some topics already on the agenda (like a report about first-hand experiences with the programming language Rust). And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.