TIL: split

I had a SQL file with 19M rows, one insert on each row, and I wanted to split it into smaller chunks. I found out that there is a split command which does exactly that:

split -l 100000 -d --additional-suffix=.sql mv-ids.sql mv-ids-

This takes mv-ids.sql file as an input and produces mv-ids-nn.sql files where each file has 100k lines. In my case nn goes from 00 to 89 and then from 9000 onwards. Good enough.

Learned today: Why DNS failed in Docker swarm containers

Our hosts are in 10.x.x.x network, and the DNS server IP is Now when we added an overlay network in Docker swarm without setting the subnet explicitly, Docker assigned the same subnet to the overlay network. When the container tried to resolve a name, it was not able to talk to the DNS server because IP packets got routed to the overlay network.

This was fixed by defining the subnets for overlay networks explicitly in another address range.

It was this GitHub issue that shed the light.

Learned today: Go templates

I built a library for loading configuration files that are actually Go templates, with some custom template functions for including values from other configuration files and loading secrets from files mounted by docker secrets.

If it turns out to be usable maybe I can open-source it some day, but for now it is proprietary.

Learned today: Apache strips Authorization header

TL;DR: CGIPassAuth On

The problem was that WordPress Rest API was not getting the HTTP basic auth information in the request. In this RunCloud setup there is first Nginx, which proxies to Apache, which proxies to FastCGI, which runs PHP ( I suppose the only reason Apache is there is to support .htaccess files).

First I suspected Nginx, because I’ve recently had issues with Nginx not always forwarding request headers to upstream host. I added  proxy_pass_request_headers on there, but it made no difference.

I created a test script that prints out the Authorization header, and indeed it was not getting all the way to PHP. Some other headers, like User-Agent were getting through.

I checked the Apache access logs, and the username was not getting printed there, so I thought that Apache is not getting the header.  I tried making the request directly to Apache, bypassing Nginx, but my test script still wasn’t getting the header. That lead me to suspect either Apache or PHP.

Finally I managed to find a Stack Overflow answer that lead me to the solution. I added CGIPassAuth On to .htaccess and bingo! Problem solved.


For a long, long time I’ve wanted to have a platform where I could store my own data wherever I want, share the data and allow applications to access it while retaining full control of who can access what. For years I’ve wanted to build such a service myself, but I’ve never got anywhere far with it.

But now, for the first time, there is a promising solution coming up. From the father of World Wide Web itself, Sir Tim Berner’s Lee: Solid.

I’ve only briefly looked at the docs, but the idea looks just like I want. I need to look at the implementation some day.

Experiences with Go

For a few months now, I’ve been programming mostly in Go. Go is a programming language developed by Google, some kind of modern version of C.

Instead of listing here all the findings, I’ll just point to the excellent summary by Sylvain Vallez. With my couple of months of experience in Go, I can fully agree with Sylvain’s points.

When I think of Go, the first thing that comes to my mind is the error handling. Probably half of my code is for error handling:

if err != nil {
   return err

But overall I’m quite satisfied with Go. It gets the job done, and I like statically typed languages. If it had generics, I might even enjoy using it.