40 Years of DSL Disasters:
From Makefile to Dockerfile

Greg Ward <greg@gerg.ca>
PyCon Canada 2017
Montreal, QC • Nov 18, 2017

I. Introduction

DSL = Domain-Specific Language

it's a large category

Some well-known examples

Some other well-known examples

(spoiler alert)

Other kinds of language: 1

on the one hand: config files

Other kinds of language: 2

on the other hand: programming languages

none of them are perfect, but the fatally flawed ones get weeded out

Hybrid languages

this seems to be hard to get right

there's more to the world DSLs than this ... but that's what this talk is limited to


I'm a build/packaging geek, so that biases my experience!

II. Unfortunate DSL Flaws

Exhibit A: Make

the grand-daddy of them all

Make: example

CC = /usr/bin/gcc
OPT = -O2

foo: main.c util.c
	$(CC) $(CFLAGS) -o $@ $^

Make is not a disaster

The real problem with make?

people who use a tool without understanding it

make is fine for C projects with a handful of source files and some man pages; that does not mean it's great for every problem

Exhibit B: autoconf

autoconf's big idea

How to do this? (ca. 1991)

solution: Bourne shell!

Oh wait... it's not portable

solution: M4 macros!

Autoconf example

                [AC_DEFINE(HAVE_FOOBAR_H, 1,
                   [Define to 1 if you have <foobar.h>.])],
                [AC_MSG_ERROR([sorry, can't do anything for you])])
  1. AC_CHECK_HEADER expands to shell code see if it can compile a C program that includes foobar.h
  2. if so: execute the expansion of AC_DEFINE
  3. if not: execute the expansion of AC_MSG_ERROR

A hard-won lesson

Exhibit C: RPM .spec files

goal: from source code, build a binary package: one little fragment of a complete working OS

so: we need to assemble metadata, variables, and fragments of shell code

RPM: example

Name: foobar
Version: 1.5.3
Source: foobar-1.5.3.tar.gz



make install

Problem 1: variables

Name: foobar
Version: 1.5.3
Source: foobar-%{version}.tar.gz

Problem 2: subroutines

%setup -q

Problem 3: setting variables

we've already seen one way to set a variable/macro... but why limit ourselves?

%define upstream_name FooBar
Name: foobar
Version: 1.5.3
Source: %{upstream_name}-%{version}.tar.gz

%setup -q -n %{upstream_name}-src

Exhibit D: Python distutils

distutils setup.py example

usage: python setup.py cmd [options...]

from distutils.core import setup    # or setuptools, nowadays

setup(name='foobar', version='1.5.3', packages=['foobar'])

setup.py counterexample

import random
from setuptools import setup

name = ('Foo' + 'Bar').lower()
version = open('version').read().strip()
packages = ['foobar']
if random.random() > 0.9:
setup(name=name, version=version, packages=packages)

Exhibit E: Dockerfile

despite the wild visions of XML or Ruby fans, custom DSLs are still a thing

Dockerfile example

FROM debian/8
RUN apt-get update && \
    apt-get upgrade && \
    apt-get install apache2 python mod_wsgi
COPY conf/myapp.conf /etc/myapp.conf
COPY myapp /usr/bin/myapp

Dockerfile problem example

be careful about making wildly popular software; even Microsoft will have to support it

FROM microsoft/nanoserver
COPY testfile.txt c:\\
RUN dir c:\

Dockerfile problem example

be careful about making wildly popular software; even Microsoft will have to support it

FROM microsoft/nanoserver
COPY testfile.txt c:\\
RUN dir c:\

...is actually equivalent to...

FROM microsoft/nanoserver
COPY testfile.txt c:\RUN dir c:\

The fix?

magic comments and dynamic syntax, of course!

# escape=`

FROM microsoft/nanoserver
COPY testfile.txt c:\
RUN dir c:\

Fun prank

slip this into your friend's Dockerfile:

# escape=a

(no, I have not tested this)

III. Wrap-Up

Red herrings

pretty sure it's wrong to blame:

lots more software fails because it is insanely ambitious than for being too modest in its aims!

My guess at the common error

Insufficiently rich syntax to accomodate future growth

The secret sauce

what do Java and Python have that autoconf and RPM don't?

(those are three ways of saying the same thing)


(have not confirmed either suspicion)


here's a bit of a grammar you know well: Python's "if" statement

if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
test: or_test ['if' or_test 'else' test] | lambdef
or_test: and_test ('or' and_test)*
[...long and complicated...]
suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE

Please don't reinvent the wheel