Do most language make false promises?

Some years ago I stumbled over this interesting article about C being the most effective of programming language and one making the least false promises. Essentially Damien Katz argues that the simplicity of C and its flaws lead to simple, fast and easy to reason about code.

C is the total package. It is the only language that’s highly productive, extremely fast, has great tooling everywhere, a large community, a highly professional culture, and is truly honest about its tradeoffs.

-Damien Katz about the C Programming language

I am Java developer most of the time but I also have reasonable experience in C, C++, C#, Groovy and Python and some other languages to a lesser extent. Damien’s article really made me think for quite some time about the languages I have been using. I think he is right in many aspects and has really good points about the tools and communities around the languages.

After quite some thought I do not completely agree with him.

My take on C

At a time I really liked the simplicity of C. I wrote gtk2hack in my spare time as an exercise and definitely see interoperability and a quick “build, run, debug”-cycle as big wins for C. On the other hand I think while it has a place in hardware and systems programming many other applications have completely different requirements.

  • A standardized ABI means nothing to me if I am writing a service with a REST/JSON interface or a standalone GUI application.
  • Portability means nothing to me if the target system(s) are well defined and/or covered by the runtime of choice.
  • Startup times mean nothing to me if the system is only started once every few months and development is still fast because of hot-code replacement or other means.
  • etc.

But I am really missing more powerful abstractions and better error handling or ressource management features. Data structures and memory management are a lot more painful than in other languages. And this is not (only) about garbage collection!

Especially C++ is making big steps in the right direction in the last few years. Each new standard release provides additional features making code more readable and less error prone. With zero cost abstractions at the core of language evolution and the secondary aim of ease of use I really like what will come to C++ in the future. And it has a very professional community, too.

Aims for the C++11 effort:

  • Make C++ a better language for systems programming and library building
  • Make C++ easier to teach and learn

-Bjarne Stroustup, A Tour of C++

What we can learn from C

Instead of looking down at C and pointing at its flaws we should look at its strengths and our own weaknesses/flaws. All languages and environments I have used to date have their own set of annoyances and gotchas.

Java people should try building simple things and having a keen eye on dependencies especially because the eco system is so rich and crowded. Also take care of ressource management – the garbage collector is only half the deal.

Scala and C++ people should take a look at ABI stability and interoperability in general. Their compile times and “build, run, debug”-cycle has much room for improvement to say the least.

C# may look at simplicity instead of wildly adding new features creating a language without opinion. A plethora of ways implementing the same stuff. Either you ban features or you have to know them all to understand code in a larger project.

Conclusion

My personal answer to the title of this blog: Yes, they make false promises. But they have a lot to offer, too.

So do not settle with the status quo of your language environment or code style of choice. Try to maintain an objective perspective and be aware of the weaknesses of the tools you are using. Most platforms improve over time and sometimes you have to re-evaluate your opinion regarding some technology.

I prefer C++ to C for some time now and did not look back yet. But I also constantly try different languages, platforms and frameworks and try to maintain a balanced view. There are often good reasons to choose one over the other for a particular project.

 

4 questions you need to ask yourself constantly while programming

Most of today’s general purpose progamming languagues come with plethora of features. Often there are different levels of abstractions and intended use cases. Some features are primarily for library designers, others ease implementation of domain specific languages and application developers use mostly another feature set.

Some language communities are discussing “language profiles / levels” to ban certain potentionally harmful constructs. The typical audience like application programmers does not need them but removing them from the language would limit its usefulness in other cases. Examples are Scala levels (a bit dated), the Google C++ Style Guide or Profiles in the C++ Core Guidelines.

In the wild

When reading other peoples code I often see novice code dealing with low-level threading. Or they go over board with templates, reflection or meta programming.

I have even seen custom ClassLoaders in Java written by normal application programmers. People are using threads when workers, tasks, actors or other more high-level abstractions would fit much better.

Especially novices seem to be unable to recognize their limits and to stay off of inappropriate and potentially dangerous features.

How do you decide what is appropriate in your situation?

Well, that is a difficult question. If you find the task at hand seems hard you should probably take a step back because:

There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors.

-Jeff Atwood

Then ask yourself some simple questions:

  1. Someone must have done it before. Have I searched thoroughly for hints or solutions?
  2. Is there a (better) library, data structure or abstraction?
  3. Do I really have to do this? There must be a better/easier way!
  4. What do I gain using feature/library/tool X and what are its costs? What about the alternatives?

Conclusion

You need some experience to recognize that you are on the wrong path, solving problems you would not even have if doing the right thing in the first place.

Experience is what you got by not having it when you needed it.

-Author Unknown

Try to know and admit your limits – there is nothing wrong with struggling to get things working but it helps to frequently check your direction by taking a step back and reflecting.

Explicit types – and when to use them

Many modern programming languages offer a way declare variables without an explicit type if the type can be inferred, either dynamically or statically. Many also allow for variables to be explicitly defined with a type. For example, Scala and C# let you omit the explicit variable type via the var keyword, but both also allow defining variables with explicit types. I’m coming from the C++ world, where “auto” is available for this purpose since the relatively recent C++11. However, people are still debating whether you should actually use it.

Pros

Herb Sutter popularised the almost-always-auto style. He advocates that using more type inference is good because it is roughly equivalent to programming against interfaces instead of implementations. He says that “Overcommitting to explicit types makes code less generic and more interdependent, and therefore more brittle and limited.” However, he also mentions that you might sometimes want to use explicit types.

Now what exactly is overcommiting here? When is the right time to use explicit types?

Cons

Opponents to implicit typing, many of them experienced veterans, often state that they want the actual type visible in the source code. They don’t want to rely on type inference being right. They want the code to explicitly state what’s going on.

At first, I figured that was just conservatism in the face of a new “scary” feature that they did not fully understand. After all, IDEs can usually infer the type on-the-fly and you can hover on a variable to let it show you the type.

For C++, the function signature is a natural boundary where you often insert explicit types, unless you want to commit to the compile time and physical dependency cost that comes with templates. Other languages, such as Groovy, do not have this trade-off and let you skip explicit types almost everywhere. After working with Groovy/Grails for a while, where the dominant style seems to be to omit types whereever possible, it dawned on me that the opponents of implicit typing have a point. Not only does the IDE often fail to show me the inferred type (even though it still works way more often than I would have anticipated), but I also found it harder to follow and modify code that did not mention explicit types. Seemingly contrary to Herb Sutter’s argument, that code felt more brittle than I had liked.

Middle-ground

As usual, the truth seems to be somewhere in the middle. I propose the following rule for when to use explicit types:

  • Explicit typing for domain-types
  • Implicit typing everywhere else

Code using types from the problem domain should be as specific as possible. There’s no need for it to be generic – it’s actually counter-productive, as otherwise the code model would be inconsistent with model of the problem domain. This is also the most important aspect to grok when reading code, so it should be explicit. The type is as important as the action on it.

On the other hand, for pure-fabrication types that do not respresent a concept in the domain, the action is important, while the type is merely a means to achieve this action. Typically, most of the elements from a language’s standard library fall into this category. All your containers, iterators, callables. Their types are merely implementation details: an associative container could be an array, or a hash-map or a tree structure. Exchanging it rarely changes the meaning of the code in the problem domain – it just changes its performance characteristics.

Containers will occasionally contain domain-types in their type. What do you do about those? I think they belong in the “everywhere else” catergory, but you should be take extra care to name the contained type when working with it – for example when declaring the variable of the for-each loop on it, or when inserting something into it. This way, the “collection of domain-type” aspect will become clear, but the specific container implementation will stay implicit – like it should.

What do you think? Is this a useful proposition for your code?

Declare war on your software

If we believe Robert Greene, life is dominated by fierce war – and he does not only refer to obvious events such as World War II or the Gulf Wars, but also to politics, jobs and even the daily interactions with your significant other.

The book

Left aside whether or not his notion corresponds to reality, it is indeed possible to apply many of the strategies traditionally employed in warfare to other fields including software development. In his book The 33 Strategies of War, Robert Greene explains his extended conception on the term war, which is not restricted to military conflicts, and describes various methods that may be utilized not only to win a battle, but also to gain advantage in everyday life. His advice is backed by detailed historic examples originating from famous military leaders like Sun Tsu, influential politicans like Franklin D. Roosevelt and even successful movie directors like Alfred Hitchcock.

Examples

While it is clear that Greene’s methods are applicable to diplomacy and politics, their application in the field of software development may seem slightly odd. Hence, I will give two specific examples from the book to explain my view.

The Grand Strategy

Alexander the Great became king of Macedon at the young age of twenty, and one of his first actions was to propose a crusade against Persia, the Greek’s nemesis. He was warned that the Persian navy was strong in the Mediterranean Sea and that he should strengthen the Greek navy so as to attack the Persians both by land and by sea. Nevertheless, he boldly set off with an army of 35,000 Greeks and marched straight into Asia Minor – and in the first encounter, he inflicted a devastating defeat on the Persians.

Now, his advisors were delighted and urged him to head into the heart of Persia. However, instead of delivering the finishing blow, he turned south, conquering some cities here and there, leading his army through Phoenicia into Egypt – and by taking Persia’s major ports, he disabled them from using their fleet. Furthermore, the Egyptians hated the Persians and welcomed Alexander, so that he was free to use their wealth of grain in order to feed his army.

Still, he did not move against the Persian king, Darius, but started to engage in politics. By building on the Persion government system, changing merely its unpopular characteristics, he was able to stabilize the captured regions and to consolidate his power. It was not before 331 B. C., two years after the start of his campaign, that he finally marched on the main Persian force.

While Alexander might have been able to defeat Darius right from the start, this success would probably not have lasted for a long time. Without taking the time to bring the conquered regions under control, his empire could easily have collapsed. Besides, the time worked in his favor: Cut off from the Egyptian wealth and the subdued cities, the Persian realm faltered.

One of Greene’s strongest points is the notion of the Grand Strategy: If you engage in a battle which does not serve a major purpose, its outcome is meaningless. Like Alexander, whose actions were all targeted on establishing a Macedonian empire, it is crucial to focus on the big picture.

It is easy to see that these guidelines are not only useful in warfare, but rather in any kind of project work – including software projects. While one has to tackle the main tasks at some point, it is important to approach it reasoned, not rashly. If anaction is not directed towards the aim of the project, one will be distracted and endager its execution by wasting resources.

The Samurai Musashi

Miyamoto Musashi, a renowned warrior and duellist, lived in Japan during the late 16th and the early 17th century. Once, he was challenged by Matashichiro, another samurai whose father and brother had already been killed by Musashi. In spite of the warning of friends that it might be a trap, he decided to oppose his enemy, however, he did prepare himself.

For his previous duels, he had arrived exorbitantly late, making his opponents lose their temper and, hence, the control over the fight. Instead, this time he appeared at the scene hours before the agreed time, hid behind some bushes and waited. And indeed, Matashichiro arrived with a small troop to ambush Musashi – but using the element of surprise, he could defeat them all.

Some time later, another warrior caught Musashi’s interest. Shishido Baiken used a kusarigama, a chain-sickle, to fight and had been undefeated so far. The chain-sickle seemed to be superior to swords: The chain offered greater range and could bind an enemy’s weapon, whereupon the sickle would deal the finishing blow. But even Baiken was thrown off his guard; Musashi showed up armed with a shortsword along with the traditional katana – and this allowed him to counter the kusarigama.

A further remarkable opponent of Musashi was the samurai Sasaki Ganryu, who wore a nodachi, a sword longer than the usual katanas. Again, Musashi changed his tactics: He faced Ganryu with an oar he had turned into a weapon. Exploiting the unmatched range of the oar, he could easily win the fight.

The characteristic that distinguished Musashi from his adversaries most was not his skill, but that he excelled at adapting his actions to his surroundings. Even though he was an outstanding swordsman, he did not hesitate to follow different paths, if necessary. Education and training facilitate becoming successful, but one has to keep an open mind to change.

Relating to software development, it does not mean that we have to start afresh all the time we begin a new project. Nevertheless, it is dangerous if one clings to outdated technologies and procedures, sometimes may be helpful to regard a situation like a child, without any assumptions. In this manner, it is probably possible to learn along the way.

Summary

Greene’s book is a very interesting read and even though in my view one should take its content with a pinch of salt, it is a nice opportunity to broaden one’s horizon. The book contains far more than I addressed in this article and I think most of its findings are indeed in one way or another applicable to everyday life.

The web is for documents

The web is intended to help a person find and understand relevant information. The primary container of information is the document. Therefore web applications should be centered around a document metaphor, not an app one.

In 1990 Tim Berners-Lee and Robert Cailliau wrote a proposal for what we call the web today:

HyperText is a way to link and access information of various kinds as a web of nodes in which the user can browse at will.

The web is a linked information system. Bret Victor states:

Information software serves the human urge to learn. A person uses information software to construct and manipulate a model that is internal to the mind — a mental representation of information.

The web is built around information. More information than we can handle. What we need to make sense of it all is understanding. The power of technology can be used to transfer and gain understanding. Understanding needs to be a first class citizen. The applications we build must be centered around it.

One way to foster understanding is to interact, to play with information. Technology can simulate a system of information so that we can form hypotheses and ask questions. Bret Victor coined the term “explorable explanations” to describe such systems.

I believe the web is perfectly suited for building explorable explanations.

The web’s container for information is the document. A document combines different forms of media (text, images, video, …) to a whole. Fortunately for us the web does not stop here. With scripting we have the possibility to interact and manipulate the information in order to gain further insight.

Most of the tools we need to create for understanding are already at our hands. What we need is a fundamental change in focus. Right now (a large part of) the web industry tries to play catch up with native. Whole frameworks try to mimic native applications like this is a virtue. Current developments want to abstract the document as far away as possible. This is not what the web was intended for. Why build an application which tries so hard to recreate a native feeling in something other than the native platform itself? Web applications should be built on the strength of the web. We should not chase a foreign metaphor.
Right now the web seems to be torn. Torn between the print era of passive documents and the shiny new world of native applications. But the web has the capability to do so much more. To concentrate on its purpose, to fill the niche. A massive niche. Understanding is a core endeavor of mankind. To quote Stephen Anderson and Karl Fast in introducing their upcoming book From Information to Understanding:

In all areas of life, we are surrounded by understanding problems.

Doug Engelbart shares a similar vision for the purpose of the personal computer per se:

By “augmenting human intellect” we mean increasing the capability of a man to approach a complex problem situation, to gain comprehension to suit his particular needs, and to derive solutions to problems.

The web is ready. The tools are ready. But are we?

Where to start: foundations

Transparent software: Making complexity understandable

“I don’t think that’s feasible”

That are not the first words we wanted to hear from our client after our presentation.

“I have seen several engineers working over a year just for the concept”

This is complex.

The world tells you that all should be simple. Make it simple. Keep it simple. It is just that simple.

Only when it is not.

Look around you. Nature is beautiful. And complex. The human is beautiful. And complex. Many systems and contexts are complex.

Look inside you. Your thoughts and emotions. Your relationships.

Look at your computer, your phone, your work. Many problems we have and solve everyday are not simple.

Sometimes we think “Oh, why can’t this be simple”. “The software should be simpler”. But KISS won’t save you. Software is broken. But not because it is not simple. We do not want simplicity. We want clarity. We want to understand. Let me elaborate.

I create software for engineers. Engineers are the people who take problems who are fine in theory and work perfectly in a controlled environment like a lab and translate them to the real world. But the real world isn’t simple or controlled. It is messy. The smallest things can blow up your house of cards of theory. These people need software to understand what happens. Through their education and their experience they know what should happen. They are the experts. But the systems and problems they work with are so complex and mostly invisible to the human eye and incomprehensible to the human brain. But today’s engineering software looks like this:

cases-colorful-colourful-2019
Options all over the place

Make no mistake. This isn’t constrained to engineering problems. Take a look around you. Nowadays there are a million sensors collecting masses of data. Your phone. Your thermostat. Your shoes. Even your tooth brush. Sensors are everywhere. We are collecting more data than ever before. This data gives us a glimpse of the complex underlying system. So we think. But why do we collect them in the first place?
Because we can. We are seeking the holy grail of wisdom. More data creates more information. More information creates more knowledge. And finally we hope that more knowledge gets us a spark of wisdom. But we are just starting out.

The course of technology

The normal way of technology goes something like this: First we are constrained. We try to push the borders. When the field is wide and open we do everything that’s possible. After a while we become more mature and use it to serve a purpose. Software is like that. Collecting data is like that. It is like an addiction. Think about it: Do you influence the data or does the data influence you? Who is in control?

But there’s hope. In order to reason about and come to our decisions we need transparent software. The dictionary defines transparent as:

transparent (adjective)

  • (of a material or article) allowing light to pass through so that objects behind can be distinctly seen
  • easy to perceive or detect
  • having thoughts, feelings, or motives that are easily perceived
  • (of an organization or its activities) open to public scrutiny
  • Physics: transmitting heat or other electromagnetic rays without distortion.
  • Computing: (of a process or interface) functioning without the user being aware of its presence.

Transparent is a tricky word. It seems to be a paradox: on the one hand it means invisible and on the other hand it means easily perceived. Both uses of the word apply to what software needs to be.

No more magic

Software has to help us understand systems and concepts. What happens and what happened. It has to make it clear, comprehensible and detectable. We need to see how the software comes to its conclusions. We need the option to overrule it. The last decision is ours. Software can help us forming a decision but it should never decide on our behalf.
Also: It gets out of our way. We don’t need any more rituals to please the software to do our bidding. Software is a tool. To be a great tool it needs to fit the problem and the person. No one wants to cut with a knife that is all blade. It should adapt to our capabilities. It should fit like a glove. It amplifies not cripples our capabilities. It is made for us. It is transparent.

That’s the goal. But how do we get there?

Maximalistic design or design with ‘Betthupferl’

Minimalistic design is a misnomer. Reducing a complex issue needs more design not less. Designing is about thinking, taking care. If we want to make complex systems understandable we need to think hard. What is the essence of the problem? What information does the expert need to evaluate a situation? All of this expertise is hidden in the heads and the daily routine of the people we design for. So we need to ask, watch and listen to them. Not direct and with a free mind. Throw your preconceptions overboard. Remove your ego. First just observe. Collect. Challenge your assumptions. When you have a good amount of information (with experience you will know when you can start but do not believe you will ever have enough), distill. Distill the essence. And then add. That little extra. The details. The cues which foster understanding.
When you stay the night in a hotel and in your white and clean bed you find a little sweet on your pillow. You are delighted. In German we call this ‘Betthupferl’.
This little extra you add is just this. The user feels cared for. He sees that someone has gone the extra mile, has thought deeply about him. The essence is not enough. You need some details. To weave the parts to together to form a whole. This can be extra information when the user needs it. This can be a shortcut when the context is right. Or an animation which guides the eye. Or or or…
Important is that it does not confuse or blur the essence. It should support. Silently, almost invisible.

Successful patterns for software developers

Be good at everything but better at something

Diversify. Knowing and working in different domains, using different programming languages and tools and handling diverse tasks enriches your creativity as a developer. It keeps your mind flexible and inspires you. Many can agree on that. But on the other hand you should find your niche, your personal joy, your home ground. Different developers have different personalities, different talents and different preferences. People shine when they work with something they like. They are more happy, more productive and more creative. But not all developers in a team or company have the same favorites. Your company should encourage you getting better at your specialty. Work on your strengths.
I like dynamic languages, others prefer static languages. Some developers like to craft desktop software, some web applications. Others concentrate on the UI or controlling robots or sensors. Some ponder over algorithms while others are happy with designing visualizations.
When your team has a common ground but everybody has strengths in different fields everybody can learn from and supplement each other. Synergy is created.

Be support, project manager, admin, …

Developers in our company wear many hats. Sometimes even literally (but that’s another story). If a developer is responsible or takes part in other roles of the project his view is widened. When he talks with customers he not only understands their needs better but also can suggest different ways or solutions. His domain knowledge grows and he can identify pain points. Working with the platforms where the application is deployed and the systems involved also strengthens his grasp of the environment the application lives in. As a project manager he learns to juggle time and scope. He learns to work with constraints and different forces pulling not only the code but the whole project.

Optimize for forgiveness

A user deleted a post accidentally. What do we normally do? We introduce a security question to establish a barrier for deletion. What’s this? We make the UX worse because we think it is the user’s fault and we need to protect him from himself. A better way would be to don’t delete the post but make it invisible. This way he can undo if he wrongly deleted the post. But what about updates? Updating a post overwrites the old content. What if this happened not on purpose? A better way would be to record the last state of the post and undo the update if necessary. Todays computers have so much memory and are so powerful that an application can be allowed to be merciful. It can forgive its users when they did something wrong and want to revert it.

It is not about you

This is one of the hardest lessons to learn. Your work is not about you. Yes, it reflects you in a way. But it does not define you, you define the work. If your code has bugs or you make a mistake and accidentally delete important data, take a deep breath and think. Get help. Plan your steps, discuss with others what to do. Tell them you made a mistake. Everybody does. Do not try to fix it yourself. Do not hide it. Focus on the problem at hand. Not your shame or feeling of guilt. (This goes hand in hand with optimizing for forgiveness)

Talk and listen

Feedback. One of the most important things in software development. Talk to your users. Listen. Again and again. Often talking 10 minutes about a task or problem can result in hours of work saved. The bug you couldn’t reproduce? It was in a different area. The feature you thought was nearly impossible? With a slight change it is much easier. This also goes for everything else. Deployments and commits. Design decisions. You are the expert in your domain, the customer in his. Act accordingly. Tell him the options he has and what will be the consequences. Don’t let him guess. He shouldn’t do your work.