Remote development with PyCharm

PyCharm is a fantastic tool for python development. One cool feature that I quite like is its support for remote development. We have quite a few projects that need to interact with special hardware, and that hardware is often not attached to the computer we’re developing on.
In order to test your programs, you still need to run it on that computer though, and doing this without tool support can be especially painful. You need to use a tool like scp or rsync to transmit your code to the target machine and then execute it using ssh. This all results in painfully long and error prone iterations.
Fortunately, PyCharm has tool support in its professional edition. After some setup, it allows you do develop just as you would on a local machine. Here’s a small guide on how to set it up with an ubuntu vagrant virtual machine, connecting over ssh. It work just as nicely on remote computers.

1. Create a new deployment configuration

In the Tools->Deployment->Configurations click the small + in the top left corner. Pick a name and choose the SFTP type.

In the “Connection” Tab of the newly created configuration, make sure to uncheck “Visible only for this project”. Then, setup your host and login information. The root path is usually a central location you have access to, like your home folder. You can use the “Autodetect” button to set this up.

For my VM, the settings look like this.

On the “Mappings” Tab, set the deployment path for your project. This would be the specific folder of your project within the root you set on the previous page. Clicking the associated “…” button here helps, and even lets you create the target folder on the remote machine if it does not exist yet.

2. Activate the upload

Now check “Tools->Deployment->Automatic Upload”. This will do an upload when you change a file, so you still need to do the initial upload manually via “Tools->Deployment->Upload to “.

3. Create a project interpreter

Now the files are synced up, but the runtime environment is not on the remote machine. Go to the “Project Interpreter” page in File->Settings and click the little gear in the top-right corner. Select “Add Remote”.

It should have the Deployment configuration you just created already selected. Once you click ok, you’re good to go! You can run and debug your code just like on a local machine.

Have fun developing python applications remotely.

The migration path for Java applets

Java applets have been a deprecated technology for a while. It has become increasingly difficult to run Java applets in modern browsers. Browser vendors are phasing out support for browser plugins based on the Netscape Plugin API (NPAPI) such as the Java plugin, which is required to run applets in the browser. Microsoft Edge and Google Chrome already have stopped supporting NPAPI plugins, Mozilla Firefox will stop supporting it at the end of 2016. This effectively means that Java applets will be dead by the end of the year.

However, it does not necessarily mean that Java applet based projects and code bases, which can’t be easily rewritten to use other technologies such as HTML5, have to become useless. Oracle recommends the migration of Java applets to Java Web Start applications. This is a viable option if the application is not required to be run embedded within a web page.

To convert an applet to a standalone Web Start application you put the applet’s content into a frame and call the code of the former applet’s init() and start() from the main() method of the main class.

Java Web Start applications are launched via the Java Network Launching Protocol (JNLP). This involves the deployment of an XML file on the web server with a “.jnlp” file extension and content type “application/x-java-jnlp-file”. This file can be linked on a web page and looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<jnlp spec="1.0+" codebase="" href="demoproject.jnlp">
  <title>Webstart Demo Project</title>
  <vendor>Softwareschneiderei GmbH</vendor>
  <j2se version="1.7+" href=""/>
  <jar href="demo-project.jar" main="true"/>
  <jar href="additional.jar"/>
 <application-desc name="My Project" main-class="com.schneide.demo.WebStartMain">
 <update check="always"/>

The JNLP file describes among other things the required dependencies such as the JAR files in the resources block, the main class and the security permissions.

The JAR files must be placed relative to the URL of the “codebase” attribute. If the application requires all permissions like in the example above, all JAR files listed in the resources block must be signed with a certificate.


The JNLP file can be linked directly in a web page and the Web Start application will be launched if Java is installed on the client operating system. Additionally, Oracle provides a JavaScript API called deployJava.js, which can be used to detect whether the required version of Java is installed on the client system and redirect the user to Oracle’s Java download page if not. It can even create an all-in-one launch button with a custom icon:

<script src=""></script>
  deployJava.launchButtonPNG = '';
  deployJava.createWebStartLaunchButton('', '1.7.0');


The Java applet technology has now reached its “end of life”, but valuable applet-based projects can be saved with relatively little effort by converting them to Web Start applications.

C++ Coroutines on Windows with the Fiber API

Last week, I had the chance to try out coroutines as a way to cooperatively interleave long tasks with event-processing. Unlike threads, where you can have interaction between between threads at any time, coroutines need to yield control explicitly, whic arguably makes synchronisation a little simpler. Especially in (legacy) systems that are not designed for concurrency. Of course, since coroutines do not run at the same time, you do not get the perks from concurrency either.

If you don’t know coroutines, think of them as functions that can be paused and resumed.

Unlike many other languages, C++ does not have built-in support for coroutines just yet. There are, however, several alternatives. On Windows, you can use the Fiber API to implement coroutines easily.

Here’s some example code of how that works:

auto coroutine=make_shared<FiberCoroutine>();
coroutine->setup([](Coroutine::Yield yield)
  for (int i=0; i<3; ++i)
    cout << "Coroutine " 
         << i << std::endl;

int stepCount = 0;
while (coroutine->step())
  cout << "Main " 
       << stepCount++ << std::endl;

Somewhat surprisingly, at least if you have never seen coroutines, this will output the two outputs alternatingly:

Coroutine 0
Main 0
Coroutine 1
Main 1
Coroutine 2
Main 2


Since fibers are not the only way to implement coroutines and since we want to keep our client code nicely insulated from the windows API, there’s a pure-virtual base class as an interface:

class Coroutine
  using Yield = std::function<void()>;
  using Run = std::function<void(Yield)>;

  virtual ~Coroutine() = default;
  virtual void setup(Run f) = 0;
  virtual bool step() = 0;

Typically, creation of a Coroutine type allocates all the resources it needs, while setup “primes” it with an inner “Run” function that can use an implementation-specific “Yield” function to pass control back to the caller, which is whoever calls step.


The implementation using fibers is fairly straight-forward:

class FiberCoroutine
  : public Coroutine
  : mCurrent(nullptr), mRunning(false)

    if (mCurrent)

  void setup(Run f) override
    if (!mMain)
      mMain = ConvertThreadToFiber(NULL);
    mRunning = true;
    mFunction = std::move(f);

    if (!mCurrent)
      mCurrent = CreateFiber(0,
        &FiberCoroutine::proc, this);

  bool step() override
    return mRunning;

  void yield()

  void run()
    while (true)
                { yield(); });
      mRunning = false;

  static VOID WINAPI proc(LPVOID data)

  static LPVOID mMain;
  LPVOID mCurrent;
  bool mRunning;
  Run mFunction;

LPVOID FiberCoroutine::mMain = nullptr;

The idea here is that the caller and the callee are both fibers: lightweight threads without concurrency. Running the coroutine switches to the callee’s, the run function’s, fiber. Yielding switches back to the caller. Note that it is currently assumed that all callers are from the same thread, since each thread that participates in the switching needs to be converted to a fiber initially, even the caller. The current version only keeps the fiber for the initial thread in a single static variable. However, it should be possible to support this by replacing the single static fiber pointer with a map that maps each thread to its associated fiber.

Note that you cannot return from the fiberproc – that will just terminate the whole thread! Instead, just yield back to the caller and either re-use or destroy the fiber.


Fiber-based coroutines are a nice and efficient way to model non-linear control-flow explicitly, but they do not come without downsides. For example, while this example worked flawlessly when compiled with visual studio, Cygwin just terminates without even an error. If you’re used to working with the visual studio debugger, it may surprise you that the caller gets hidden completely while you’re in the run function. The run functions stack completely replaces the callers stack until you call yield(). This means that you cannot find out who called step(). On the other hand, if you’re actually doing a lot of processing in the run function, this is quite nice for profiling, as the “processing” call tree seemingly has its own root in the call-tree.

I just wish the visual studio debugger had a way to view the states of the different fibers like it has for threads.


  • On Linux, you can use the ucontext.
  • Visual Studio 2015 also has another, newer, implementation.
  • Coroutines can be implemented using threads and condition-variables.
  • There’s also Boost.Coroutine, if you need an independent implementation of the concept. From what I gather, they only use Fibers optionally, and otherwise do the required “trickery” themselves. Maybe this even keeps the caller-stack visible – it is certainly worth exploring.
  • Generating a spherified cube in C++

    In my last post, I showed how to generate an icosphere, a subdivided icosahedron, without any fancy data-structures like the half-edge data-structure. Someone in the reddit discussion on my post mentioned that a spherified cube is also nice, especially since it naturally lends itself to a relatively nice UV-map.

    The old algorithm

    The exact same algorithm from my last post can easily be adapted to generate a spherified cube, just by starting on different data.


    After 3 steps of subdivision with the old algorithm, that cube will be transformed into this:


    Slightly adapted

    If you look closely, you will see that the triangles in this mesh are a bit uneven. The vertical lines in the yellow-side seem to curve around a bit. This is because unlike in the icosahedron, the triangles in the initial box mesh are far from equilateral. The four-way split does not work very well with this.

    One way to improve the situation is to use an adaptive two-way split instead:

    Instead of splitting all three edges, we’ll only split one. The adaptive part here is that the edge we’ll split is always the longest that appears in the triangle, therefore avoiding very long edges.

    Here’s the code for that. The only tricky part is the modulo-counting to get the indices right. The vertex_for_edge function does the same thing as last time: providing a vertex for subdivision while keeping the mesh connected in its index structure.

    subdivide_2(ColorVertexList& vertices,
      TriangleList triangles)
      Lookup lookup;
      TriangleList result;
      for (auto&& each:triangles)
        auto edge=longest_edge(vertices, each);
        Index mid=vertex_for_edge(lookup, vertices,
          each.vertex[edge], each.vertex[(edge+1)%3]);
          mid, each.vertex[(edge+2)%3]});
          mid, each.vertex[(edge+1)%3]});
      return result;

    Now the result looks a lot more even:

    Note that this algorithm only doubles the triangle count per iteration, so you might want to execute it twice as often as the four-way split.


    Instead of using this generic of triangle-based subdivision, it is also possible to generate the six sides as subdivided patches, as suggested in this article. This approach works naturally if you want to have seams between your six sides. However, that approach is more specialized towards this special geometry and will require extra “stitching” if you don’t want seams.


    The code for both the icosphere and the spherified cube is now on github:

    My favorite Unix tool

    Awk is a little language designed for the processing of lines of text. It is available on every Unix (since V3) or Linux system. The name is an acronym of the names of its creators: Aho, Weinberger and Kernighan.

    Since I spent a couple of minutes to learn awk I have found it quite useful during my daily work. It is my favorite tool in the base set of Unix tools due to its simplicity and versatility.

    Typical use cases for awk scripts are log file analysis and the processing of character separated value (CSV) formats. Awk allows you to easily filter, transform and aggregate lines of text.

    The idea of awk is very simple. An awk script consists of a number of patterns, each associated with a block of code that gets executed for an input line if the pattern matches:

    pattern_1 {
        # code to execute if pattern matches line
    pattern_2 {
        # code to execute if pattern matches line
    # ...
    pattern_n {
        # code to execute if pattern matches line

    Patterns and blocks

    The patterns are usually regular expressions:

    /error|warning/ {
        # executed for each line, which contains
        # the word "error" or "warning"
    /^Exception/ {
        # executed for each line starting
        # with "Exception"

    There are some special patterns, namely the empty pattern, which matches every line …

        # executed for every line

    … and the BEGIN and END patterns. Their blocks are executed before and after the processing of the input, respectively:

    BEGIN {
        # executed before any input is processed,
        # often used to initialize variables
    END {
        # executed after all input has been processed,
        # often used to output an aggregation of
        # collected values or a summary

    Output and variables

    The most common operation within a block is the print statement. The following awk script outputs each line containing the string “error”:

    /error/ { print }

    This is basically the functionality of the Unix grep command, which is filtering. It gets more interesting with variables. Awk provides a couple of useful built-in variables. Here are some of them:

    • $0 represents the entire current line
    • $1$n represent the 1…n-th field of the current line
    • NF holds the number of fields in the current line
    • NR holds the number of the current line (“record”)

    By default awk interprets whitespace sequences (spaces and tabs) as field separators. However, this can be changed by setting the FS variable (“field separator”).

    The following script outputs the second field for each line:

    { print $2 }


    John 32 male
    Jane 45 female
    Richard 73 male



    And this script calculates the sum and the average of the second fields:

        sum += $2
    END {
        print "sum: " sum ", average: " sum/NR


    sum: 150, average: 50

    The language

    The language that can be used within a block of code is based on C syntax without types and is very similar to JavaScript. All the familiar control structures like if/else, for, while, do and operators like =, ==, >, &&, ||, ++, +=, … are there.

    Semicolons at the end of statements are optional, like in JavaScript. Comments start with a #, not with //.

    Variables do not have to be declared before usage (no ‘var’ or type). You can simply assign a value to a variable and it comes into existence.

    String concatenation does not have an explicit operator like “+”. Strings and variables are concatenated by placing them next to each other:

    "Hello " name ", how are you?"
    # This is wrong: "Hello" + name + ", how are you?"

    print is a statement, not a function. Parentheses around its parameter list are optional.


    Awk provides a small set of built-in functions. Some of them are:

    length(string), substr(string, index, count), index(string, substring), tolower(string), toupper(string), match(string, regexp).

    User-defined functions look like JavaScript functions:

    function min(number1, number2) {
        if (number1 < number2) {
            return number1
        return number2

    In fact, JavaScript adopted the function keyword from awk. User-defined functions can be placed outside of pattern blocks.

    Command-line invocation

    An awk script can be either read from a script file with the -f option:

    $ awk -f myscript.awk data.txt

    … or it can be supplied in-line within single quotes:

    $ awk '{sum+=$2} END {print "sum: " sum " avg: " sum/NR}' data.txt


    I hope this short introduction helped you add awk to your toolbox if you weren’t familiar with awk yet. Awk is a neat alternative to full-blown scripting languages like Python and Perl for simple text processing tasks.

    Generating an Icosphere in C++

    If you want to render a sphere in 3D, for example in OpenGL or DirectX, it is often a good idea to use a subdivided icosahedron. That often works better than the “UVSphere”, which means simply tesselating a sphere by longitude and latitude. The triangles in an icosphere are a lot more evenly distributed over the final sphere. Unfortunately, the easiest way, it seems, is to generate such a sphere is to do that in a 3D editing program. But to load that into your application requires a 3D file format parser. That’s a lot of overhead if you really need just the sphere, so doing it programmatically is preferable.

    At this point, many people will just settle for the UVSphere since it is easy to generate programmatically. Especially since generating the sphere as an indexed mesh without vertex-duplicates further complicates the problem. But it is actually not much harder to generate the icosphere!
    Here I’ll show some C++ code that does just that.

    C++ Implementation

    We start with a hard-coded indexed-mesh representation of the icosahedron:

    struct Triangle
      Index vertex[3];
    using TriangleList=std::vector<Triangle>;
    using VertexList=std::vector<v3>;
    namespace icosahedron
    const float X=.525731112119133606f;
    const float Z=.850650808352039932f;
    const float N=0.f;
    static const VertexList vertices=
      {-X,N,Z}, {X,N,Z}, {-X,N,-Z}, {X,N,-Z},
      {N,Z,X}, {N,Z,-X}, {N,-Z,X}, {N,-Z,-X},
      {Z,X,N}, {-Z,X, N}, {Z,-X,N}, {-Z,-X, N}
    static const TriangleList triangles=

    Now we iteratively replace each triangle in this icosahedron by four new triangles:


    Each edge in the old model is subdivided and the resulting vertex is moved on to the unit sphere by normalization. The key here is to not duplicate the newly created vertices. This is done by keeping a lookup of the edge to the new vertex it generates. Note that the orientation of the edge does not matter here, so we need to normalize the edge direction for the lookup. We do this by forcing the lower index first. Here’s the code that either creates or reused the vertex for a single edge:

    using Lookup=std::map<std::pair<Index, Index>, Index>;
    Index vertex_for_edge(Lookup& lookup,
      VertexList& vertices, Index first, Index second)
      Lookup::key_type key(first, second);
      if (key.first>key.second)
        std::swap(key.first, key.second);
      auto inserted=lookup.insert({key, vertices.size()});
      if (inserted.second)
        auto& edge0=vertices[first];
        auto& edge1=vertices[second];
        auto point=normalize(edge0+edge1);
      return inserted.first->second;

    Now you just need to do this for all the edges of all the triangles in the model from the previous interation:

    TriangleList subdivide(VertexList& vertices,
      TriangleList triangles)
      Lookup lookup;
      TriangleList result;
      for (auto&& each:triangles)
        std::array<Index, 3> mid;
        for (int edge=0; edge<3; ++edge)
          mid[edge]=vertex_for_edge(lookup, vertices,
            each.vertex[edge], each.vertex[(edge+1)%3]);
        result.push_back({each.vertex[0], mid[0], mid[2]});
        result.push_back({each.vertex[1], mid[1], mid[0]});
        result.push_back({each.vertex[2], mid[2], mid[1]});
        result.push_back({mid[0], mid[1], mid[2]});
      return result;
    using IndexedMesh=std::pair<VertexList, TriangleList>;
    IndexedMesh make_icosphere(int subdivisions)
      VertexList vertices=icosahedron::vertices;
      TriangleList triangles=icosahedron::triangles;
      for (int i=0; i<subdivisions; ++i)
        triangles=subdivide(vertices, triangles);
      return{vertices, triangles};

    There you go, a customly subdivided icosphere!


    Of course, this implementation is not the most runtime-efficient way to get the icosphere. But it is decent and very simple. Its performance depends mainly on the type of lookup used. I used a map instead of an unordered_map here for brevity, only because there’s no premade hash function for a std::pair of indices. In pratice, you would almost always use a hash-map or some kind of spatial structure, such as a grid, which makes this method a lot tougher to compete with. And certainly feasible for most applications!

    The general pattern

    The lookup-or-create pattern used in this code is very useful when creating indexed-meshes programmatically. I’m certainly not the only one who discovered it, but I think it needs to be more widely known. For example, I’ve used it when extracting voxel-membranes and isosurfaces from volumes. It works very well whenever you are creating your vertices from some well-defined parameters. Usually, it’s some tuple that describes the edge you are creating the vertex on. This is the case with marching cubes or marching tetrahedrons. It can, however, also be grid coordinates if you sparsely generate vertices on a grid, for example when meshing heightmaps.

    ECMAScript 6 is coming

    ECMAScript is the standardized specification of the scripting language that is the basis for JavaScript implementations in web browsers.

    ECMAScript 5, which was released in 2009, is in widespread use today. Six years later in 2015 ECMAScript 6 was released, which introduced a lot of new features, especially additions to the syntax.

    However, browser support for ES 6 needed some time to catch up.
    If you look at the compatibility table for ES 6 support in modern browsers you can see that the current versions of the main browsers now support ES 6. This means that soon you will be able to write ES 6 code without the necessity of a ES6-to-ES5 cross-compiler like Babel.

    The two features of ES 6 that will change the way JavaScript code will be written are in my opinion: the new class syntax and the new lambda syntax with lexical scope for ‘this’.

    Class syntax

    Up to now, if you wanted to define a new type with a constructor and methods on it, you would write something like this:

    var MyClass = function(a, b) {
    	this.a = a;
    	this.b = b;
    MyClass.prototype.methodA = function() {
    	// ...
    MyClass.prototype.methodB = function() {
    	// ...
    MyClass.prototype.methodC = function() {
    	// ...

    ES 6 introduces syntax support for this pattern with the class keyword, which makes class definitions shorter and more similar to class definitions in other object-oriented programming languages:

    class MyClass {
    	constructor(a, b) {
    		this.a = a;
    		this.b = b;
    	methodA() {
    		// ...
    	methodB() {
    		// ...
    	methodC() {
    		// ...

    Arrow functions and lexical this

    The biggest annoyance in JavaScript were the scoping rules of this. You always had to remember to rebind this if you wanted to use it inside an anonymous function:

    var self = this;
    this.numbers.forEach(function(n) {

    An alternative was to use the bind method on an anonymous function:

    this.numbers.forEach(function(n) {

    ES 6 introduces a new syntax for lambdas via the “fat arrow” syntax:

    this.numbers.forEach((n) => this.doSomething(n));

    These arrow functions do not only preserve the binding of this but are also shorter. Especially if the function body consists of only a single expression. In this case the curly braces and and the return keyword can be omitted:

    // ES 5
    numbers.filter(function(n) { return isPrime(n); });
    // ES 6
    numbers.filter((n) => isPrime(n));

    Wide-spread support for ECMAScript 6 is just around the corner, it’s time to familiarize yourself with it if you haven’t done so yet. A nice overview of the new features can be found at