skip to main content.

posts about www.

four days ago, arch linux switched to openssl 1.1.0. openssl 1.1.0 was originally released at the end of last august, but since it has some breaking api changes, it's only slowly creeping into new linux distributions.

this also means that i can finally test my let's encrypt library, let's encrypt ansible role and ocspbot against openssl 1.1.0. the let's encrypt code worked out of the box (i've already incorporated a change somewhen earlier, even without being able to properly test it), but ocspbot needed a bit more work. there's a command line syntax change between 1.0.x and 1.1.0 when specifying http headers to ocsp calls; the old syntax was -header name value, the new one is -header name=value. so i had to add a version detection (i.e. parsing the output of openssl version) to use the correct syntax depending on the used version. but now it works with both openssl 1.0.x and 1.1.0!

using openssl 1.1.0 on my server also allowed me to use x25519, using daniel j. bernstein's curve25519 in edwards form, for secret key negotation (i.e. ephemeral diffie-hellman). using it in nginx is pretty easy:

ssl_ecdh_curve X25519:secp521r1:secp384r1;

this uses x25519 as the default curve/key exchange, followed by the fallsbacks using ecdhe with a 521-bit nist curve and then a 384-bit nist curve as a third fallback. (btw, note the uppercase x in x25519 — if you use the lowercase variant, nginx won't load the config.) the third curve is the only one supported by almost every browser; only a few support the 521-bit one, and right now only chrome supports x25519.

assume you want to allow users (or programs) to upload files/data/... to your website without having to write a script/cgi/... which handles the uploading. something very simple, which just stores the files somewhere so you can analyze them later. this is for example very useful to create a reporting endpoint for a content security policy without using specialized software.

if you use nginx as your webserver, there's a simple solution for this. the idea is to use the client_body_in_file_only directive, which allows you to dump uploads to disk to pass the filename to a reverse proxy, instead of asking nginx to cache the upload and pass it on to the reverse proxy, so that it looks like a regular post/put to the reverse proxy.

unfortunately, this doesn't work if you use a return xxx; instead of proxy_pass yyy;, which would have been my preferred solution. but there's a little trick: you can ask nginx to also listen on another port, say 4000, and simply return a fixed message there. then, for the main listener (on ports 80/443), you use client_body_in_file_only directive combined with proxy_pass http://127.0.0.1:4000. this looks as follows:

server {
    listen 127.0.0.1:4000;

    location / {
        return 200 "Thank you for your report.\n";
    }
}

limit_req_zone $binary_remote_addr zone=peripzone:10m rate=5r/m;

server {
    listen *:80;

    location = /csp-reporting {
        # We just allow POST actions. Add PUT if you want to
        # support PUT as well. (Not needed for CSP reports.)
        limit_except POST {
            deny all;
        }

        # Where to store the files on disk
        client_body_temp_path      /var/www/reports/csp/;
        # Store the file on disk, and don't delete it, no matter
        # what the proxy returns.
        client_body_in_file_only   on;
        # Store at most 64k on disk. That should be sufficient
        # for CSP reports.
        client_body_buffer_size    64K;
        client_max_body_size       64K;
        # Give the client 10 seconds to upload.
        client_body_timeout        10s;

        # Do rate limiting
        limit_req                  zone=peripzone burst=20 nodelay;

        # Now proxy to the small internal server we started
        # above, and don't pass the uploaded file to it.
        proxy_set_body             off;
        proxy_pass                 http://127.0.0.1:4000/;
    }
}

note that i added rate limiting (see the ngx_http_limit_req_module module), allowing on average five uploads per second for remote ips, with bursts up to 10 uploads. see the ngx_http_limit_req_module module's documentation for more information on adding more rate limiting. you can for example also add limits per server, instead of just one per remote ip.

posted in: www
tags:
places:

introduction

classically, revokation of certificates was accomplished with certificate revokation lists (crls). the idea was that browsers regularly download crls from the certificate authorities (cas) and check whether certificates they see are on the list. this doesn't scale well, though. nowadays, there are many cas trusted by browsers in their default configuration, and crls tend to get huge.

a better solution is the online certificate status protocol (ocsp): a browser, when encountering a new certificate, asks the ocsp server of the browser (the url for it is contained in the certificate) whether the certificate is still valid. this has several downsides as well: first, ocsp servers are not always reliable. if a browser cannot connect to it (or doesn't get a reply), what should it do? deny access to the site? besides that, there's another large downside: privacy. the ocsp server knows which page you are visiting, because the browser tells it by asking whether its certificate is valid.

ocsp stapling was invented to improve upon this: the idea is that the webserver itself asks the ocsp server for the status of its certificate, and delivers the answer together with the certificate to the connecting browser. as the ocsp response is signed by the cas certificate, the browser can verify that the response is valid. also, the expiration time of ocsp responses is much less than the one for certificates, so if a certificate is revoked, existing ocsp responses will only be valid for a couple of more days.

this is pretty good already, except that a malicious webserver could simply not send the ocsp response with its certificate. if a browser cannot contact the ocsp server itself, it has no way to know whether the certificate is revoked or not. to overcome this, ocsp must-staple was invented. this is a flag in the certificate itself which says that the certificate is only valid with a valid and good ocsp response. so if a browser encounters a certificate with this flag, and the webserver isn't ocsp stapling, the browser knows that something is fishy.

unfortunately, there are some downsides. first, most the most common webservers for linux, apache and nginx, while having ocsp stapling support, do in some situations send replies without ocsp stapling. if the certificate has the ocsp must-staple flag set, these answers result in error pages shown in browsers. and that's something you really want to avoid, that visitors of your page thing there's something bad happening.

fortunately, at least for nginx, you can specify a file containing an ocsp response directly with the ssl_stapling_file directive. unfortunately, you have to make sure you always have a good and valid ocsp response at that place, and reload nginx in case the response is updated. other programs allow to specify an ocsp response in a similar way, such as exim with the tls_ocsp_file directive, and thus have the same problem. to solve this problem, i've started creating ocsp bot:

ocsp bot

ocsp bot is a python script which should be called frequently (as in: once per hour or so), which checks a set of x.509 certificates to obtain up-to-date ocsp responses. in case the current ocsp responses will expire soon, or aren't there, it will try to get a new response. it will only copy the new response to the correct place if it is valid and good. calling it frequently will ensure that in case of problems getting a new response, it will retry every hour (or so) until a good and valid response could have been obtained. so a user response is only necessary if the process fails several times in a row.

ocsp bot will signal with its exit code whether responses have been updated, allowing to reload/restart the corresponding service to use the new response.

you can install ocsp bot with pip install ocsp from pypi.

integration with ansible

i'm using ansible to configure my server. to copy certificates and obtain ocsp responses, i'm using a custom role.

the ansible tasks for the role are as follows. ocsp bot is installed in /var/www/ocsp:

- name: Create OCSP log folder
  file: dest=/var/www/ocsp/logs state=directory
- name: Create OCSP response folder
  file: dest=/var/www/ocsp/responses state=directory
- name: Install pyyaml
  pip: name=pyyaml
- name: Install OCSP response utility
  copy: src=ocspbot.py dest=/var/www/ocsp/ocspbot.py mode=0755
  # ocspbot.py is ocspbot/__main__.py from https://github.com/felixfontein/ocspbot/
- name: Install OCSP bash script
  template: src=ocspbot.sh.j2 dest=/var/www/ocsp/ocspbot.sh mode=0755
- name: Install OCSP response utility configurations
  template: src=ocspbot.yaml.j2 dest=/var/www/ocsp/ocspbot-{{ item.key }}.yaml
  with_dict: "{{ certificates }}"
- name: Install OCSP response cronjob
  cron: name="Update OCSP responses" hour=* minute=0 job=/var/www/ocsp/ocspbot.sh state=present

the variable certificates is defined as follows:

certificates:
  nginx:
    domains:
    - example.com
    - example.net
    reload:
    - nginx
    key_owner: root
    key_group: root
    key_mode: "0400"
  mailserver:
    domains:
    - mail.example.com
    reload:
    - dovecot
    - exim
    key_owner: root
    key_group: exim
    key_mode: "0440"

The template for ocspbot.sh:

#!/bin/bash
RC=0
{% for name, data in certificates|dictsort %}

# Renew OCSP responses for {{ name }}
/var/www/ocsp/ocspbot.py /home/ocsp/ocspbot-{{ name }}.yaml
RESULT=$?
if [ $RESULT -gt 0 ]; then
{%   for service in data.reload %}
    systemctl reload {{ service }}
{%   endfor %}
elif [ $RESULT -lt 0 ]; then
    RC=1
fi
{% endfor %}

exit $RC

the template for the configuration yaml files:

make_backups: True

minimum_validity: 3d
minimum_validity_percentage: 42.8

ocsp_folder: /var/www/ocsp/responses
output_log: /var/www/ocsp/logs/{{ item.key }}-{year}{month}{day}-{hour}{minute}{second}.log

domains:
{% for domain in item.value.domains|sort %}
  {{ domain }}:
    cert: /var/www/certs/{{ domain }}.pem
    chain: /var/www/certs/{{ domain }}-chain.pem
    rootchain: /var/www/certs/{{ domain }}-rootchain.pem
    ocsp: {{ domain }}.ocsp-resp
{% endfor %}

the certificates are copied with the following ansible tasks:

- name: copy private keys
  copy: src=keys/{{ item.1 }}.key dest=/var/www/keys/{{ item.1 }}.key owner={{ item.0.value.key_owner }} group={{ item.0.value.key_group }} mode={{ item.0.value.key_mode }}
  with_dependent:
  - "certificates"
  - "item.0.value.domains"
  notify: update OCSP responses
- name: copy certificates
  copy: src=keys/{{ item.1 }}{{ item.2 }} dest=/var/www/certs/{{ item.1 }}{{ item.2 }} owner=root group=root mode=0444
  with_dependent:
  - "certificates"
  - "item.0.value.domains"
  - '["-rootchain.pem", "-fullchain.pem", "-chain.pem", ".pem"]'
  notify: update OCSP responses

(here, the dependent loop lookup plugin is used.)

the handler update OCSP responses is defined as follows:

- name: update OCSP responses
  command: /var/www/ocsp/ocspbot.sh
  register: result
  failed_when: result.rc != 0
  notify:
  - reload nginx
  - reload exim
  - reload dovecot

i'm using this setup for some weeks now, and it seems to work fine. so far, i'm not using ocsp must-staple certificates (except for some test subdomains). if everything seems to be fine for some time, i'll switch to ocsp must-staple certificates.

today, i've added two new plugins to nikola: sidebar and static tag cloud. these, together with filetreesubs, a doit-based file tree synchronization and text-based substitution tool, allow to use nikola to create a more dynamic-looking blog with a sidebar having current information, without the need to rebuild every single page every time something changes. (which can take a very long time for a large blog such as spielwiese).

the plugins create html fragments and, for the tag clouds, css files, which have to be included in all generated blog pages. one way to do this is to use javascript, but that wouldn't yield a proper static blog as i imagine it. the sidebar and the tag cloud should always be there, and not depend on javascript being enabled. (i for myself use noscript and javascript in my browser is off by default.)

for spielwiese, a small tag cloud is created (one per language) and included in the sidebar. a large tag cloud is created and included in the tag overview page (chde version). also, a small and large place cloud is created and included in the sidebar and place overview page, respectively.

the configuration for the static tag cloud plugin looks as follows:

RENDER_STATIC_TAG_CLOUDS = {
    # Small tag cloud
    'small': {
        'name': 'tcs-{0}',
        'filename': 'tagcloud-{0}.inc',
        'taxonomy_type': 'tag',
        'style_filename': 'assets/css/tagcloud-{0}-small.css',
        'max_number_of_levels': 15,
        'max_tags': 40,
        'minimal_number_of_appearances': 5,
        'colors': ((0.4,0.4,0.4), (1.0,1.0,1.0)),
        'background_colors': ((0.133, 0.133, 0.133), ),
        'border_colors': ((0.2, 0.2, 0.2), ),
        'font_sizes': (6, 20),
        'round_factor': 0.6,
    },
    # Large tag cloud
    'large': {
        'name': 'tcl-{0}',
        'filename': 'tagcloud-{0}-large.inc',
        'taxonomy_type': 'tag',
        'style_filename': 'assets/css/tagcloud-{0}-large.css',
        'max_number_of_levels': 100,
        'minimal_number_of_appearances': 3,
        'colors': ((0.25,0.25,0.25), (1.0,1.0,1.0)),
        'background_colors': ((0.133, 0.133, 0.133), ),
        'border_colors': ((0.2, 0.2, 0.2), ),
        'font_sizes': (8, 35),
        'round_factor': 0.3,
    },
    # Small place cloud
    'places-small': {
        'name': 'pcs-{0}',
        'filename': 'placecloud-{0}.inc',
        'taxonomy_type': 'place',
        'style_filename': 'assets/css/placecloud-{0}-small.css',
        'max_number_of_levels': 15,
        'max_tags': 40,
        'minimal_number_of_appearances': 3,
        'colors': ((0.4,0.4,0.4), (1.0,1.0,1.0)),
        'background_colors': ((0.133, 0.133, 0.133), ),
        'border_colors': ((0.2, 0.2, 0.2), ),
        'font_sizes': (6, 20),
        'round_factor': 0.6,
    },
    # Large place cloud
    'places-large': {
        'name': 'pcl-{0}',
        'filename': 'placecloud-{0}-large.inc',
        'taxonomy_type': 'place',
        'style_filename': 'assets/css/placecloud-{0}-large.css',
        'max_number_of_levels': 100,
        'minimal_number_of_appearances': 2,
        'colors': ((0.25,0.25,0.25), (1.0,1.0,1.0)),
        'background_colors': ((0.133, 0.133, 0.133), ),
        'border_colors': ((0.2, 0.2, 0.2), ),
        'font_sizes': (8, 35),
        'round_factor': 0.3,
    },
}

the generated css file for the large tag cloud can be found here; the generated html fragments aren't uploaded as filetreesubs doesn't copy them to the output folder.

the configuration for filetreesubs looks as follows:

source: output-spielwiese
destination: final-spielwiese

# Substitutions
substitutes:
  # For all HTML pages: include sidebar
  '.*\.html':
    '<!-- include:sidebar-en -->':
      file: sidebar-en.inc
    '<!-- include:sidebar-chde -->':
      file: sidebar-chde.inc
  # For specific pages, also include tag/place clouds
  'tag/index.html':
    '<!-- include:tagcloud:en:large -->':
      file: tagcloud-en-large.inc
  'place/index.html':
    '<!-- include:placecloud:en:large -->':
      file: placecloud-en-large.inc
  'chde/schlagwort/index.html':
    '<!-- include:tagcloud:chde:large -->':
      file: tagcloud-chde-large.inc
  'chde/ort/index.html':
    '<!-- include:placecloud:chde:large -->':
      file: placecloud-chde-large.inc

# The substitution chains allow the sidebar to include
# the small tag and place clouds.
substitute_chains:
- template: sidebar-en.inc
  substitutes:
    '<!-- include:tagcloud:en -->':
      file: tagcloud-en.inc
    '<!-- include:placecloud:en -->':
      file: placecloud-en.inc
- template: sidebar-chde.inc
  substitutes:
    '<!-- include:tagcloud:chde -->':
      file: tagcloud-chde.inc
    '<!-- include:placecloud:chde -->':
      file: placecloud-chde.inc

# Create index.html files in all folders which don't have one yet.
create_index_filename: index.html
create_index_content: |
  <!DOCTYPE html>
  <html lang="en">
    <head>
      <title>there's nothing to see here.</title>
      <meta name="robots" content="noindex">
      <meta http-equiv="refresh" content="0; url=..">
    </head>
    <body style="background-color:black; color:white;">
      <div style="position:absolute; top:0; left:0; right:0; bottom:0;">
        <div style="width:100%; height:100%; display:table;">
          <div style="display:table-cell; text-align:center; vertical-align:middle;">
            there's nothing to see here. go <a href=".." style="color:#AAA;">here</a> instead.
          </div>
        </div>
      </div>
    </body>
  </html>

# Everything is UTF-8.
encoding: utf-8

# I want to be able to run different things in parallel.
doit_config:
  dep_file: '.doit-spielwiese-subs.db'

this allows me to use html-style comments such as <!-- include:tagcloud:en --> to indicate where the html fragments should be included. i also create index files in foldes which otherwise would be empty, such as /photos/ (see here for how it looks). the result is, from my point of view, a much more polished version of the blog than the raw version produced by nikola without postprocessing.

this should give you an idea on how to produce a similar result with nikola, my plugins and filetreesubs.

posted in: www
tags:
places:

it’s more than eight years ago that i started spielwiese.

so far, spielwiese was mostly a black and white, lowercase english experience. until now. using nikola‘s multilingual capabilities, i decided to try translating parts of spielwiese to züritüütsch, a swiss german dialect spoken in the canton of zürich, in which i’ve now spent more than eight years of my life.

since züritüütsch isn’t my native language and i’m still far from understanding and speaking it perfectly (though understanding is much better than speaking), i’ll surely make mistakes, though i hope most will be caught by my wife before setting them live :)

this is yet another experiment, and i’ll hope some of you will like it.

after already revamping math and musikwiese earlier this month, it’s time to do spielwiese as well. and here we go! spielwiese now adjusts to all window widths, including mobile phones. and the gallery looks good again, too. (i had to screw with it when adjusting spielwiese for mobile phones the first time.)

the only things missing are a proper popup gallery for mobile phones, and as in the other blogs: comments. so sorry, there’s no way to add a comment at the moment, except by writing me an email.

posted in: www
tags:
places:

over the last weeks, i’ve been starting to convert my blogs to nikola, a static blog generator named after nikola tesla. so far, i’ve finished my math blog and musikwiese. i also got rid of the fixed width layout. spielwiese itself will take a bit more work, mostly because of the wordpress plugins i wrote and heavily rely on throughout this blog.

to convert my blogs, i had to add some changes to nikola. most of the core changes are now contained in nikola itself or are close to be integrated (except xhtml support). i also worked a lot on improving its wordpess import, which so far wasn’t very helpful from my point of view. for that, i created a wordpress page compiler for nikola which essentially does the same to posts as wordpress’ formatter does. that part is still not completely done, and since it is a pure plugin, i haven’t released it yet. (that will happen later, hopefully still this year.)

the main disadvantage of the whole conversion process is currently that there’s no more possibility to add comments. all comments written so far are preserved, but currently there’s no support for adding new ones. i’ll try to figure out a good way to do that later. the “standard way” to do that, i.e. using something like disqus, is not acceptable for me, and even though there are free alternatives for self-hosting, i don’t like to rely on javascript for such things. for the moment, you have to send me an email if you want to add a comment to musikwiese or my math blog.

i have several wordpress installations running on my servers. among them, there is one instance which is not publicly visible. it uses dates which are several decades in the past.
everything was working well when i last worked on that project, which is quite some time ago. but yesterday, when looking at it another time, i noticed something odd. the dates were all replaced with the current date (and time). wtf?
now, some debugging later, i found the culprit. that wordpress installation is in german (the only german one i have), and somewhere inside wordpress, a function called date_i18n is used in that case to translate a timestamp into a date, time, or whatever is requested. unfortunately (for me), this function (in Wordpress/wp-includes/functions.php) contains a “sanity check” for “php 5.1.0-”. in case the unix timestamp is negative, it is set to current time. and obviously, any date before january 1, 1970 will be turned into today.
commenting out these lines (or removing them) fixes the problem.

posted in: www
tags:
places:

as you might have noticed, all videos embedded from youtube or vimeo got replaced by a disclaimer, stating that i do not want to bear the legal risks of including them anymore. including something you didn’t create yourself is kind of russian roulette. if someone decides to send you a abmahnung, trouble begins. and usually, money goes. lots of money. even if what you do is legal, you’ll have to invest quite some money to make your point. i don’t really want to do this for these videos, so i changed my plugin to let it spit out disclaimers with links instead of embedding* the video. i’m sorry for this, in particular this essentially ruins musikwiese.

*the videos weren’t embedded directly for some time anyway, see here, but i guess that wouldn’t really make difference…

in this post, i want to explain how to create a simple neat javascript/css “fullscreen” image viewer, or more concretely, the one i implemented myself here on spielwiese. feel free to re-use the code. it is not garuanteed to work everywhere, but it works fine with firefox, konqueror and chromium/chrome, to my knowledge, and probably also with safari (as it is based on konqueror), and hopefully with opera and more modern versions of internet explorer. i didn’t include any hacks to make it work under older versions of internet explorer or other browsers, but feel free to enhance my version and add a link to yours in the comments here.
i assume that so far, you include your images using

<a href=”path/to/image.jpeg”><img src=”path/to/image-thumbnail.jpeg” /></a>.

this displays a thumbnail (called path/to/image-thumbnail.jpeg) and allows the user to click it, to be forwarded to the real image (called path/to/image.jpeg). maybe you also use an alt tag, or specify the size of the thumbnail. that doesn’t really matter.
the idea is to add a javascript onclick event to the <a> tag, which will become active once the link (i.e. the image) is clicked and show the “popup”. in case the user disabled javascript, everything will fall back to the old behaviour: the user will be redirected to the image and no harm is done by the addition of the new code. the above html code has to be modified as follows:

<a href=”path/to/image.jpeg” onclick=”showPicture(‘path/to/image.jpeg’)“><img src=”path/to/image-thumbnail.jpeg” /></a>

the addition is printed in bold. the drawback of this method is that you have to replicate the image’s name, but then, you already had to do that for the thumbnail, and if you have automated the whole process, changing your script to insert this part shouldn’t be any problem.
so now let me show you how to implement the showPicture() function.

 1<script type="text/javascript">
 2//<![CDATA[
 3function showPicture(url) {
 4  document.body.style.overflow = "hidden";
 5  el = document.createElement("div");
 6  el.style.cssText = "background: black; color: white; position: fixed; left: 0px; top: 0px; width: 100%; height: 100%; text-align: center; z-index: 1000;";
 7  el.onclick = Function("this.parentNode.removeChild(this); document.body.style.overflow = 'visible';");
 8  var code = '<div style="width: 100%; height: 100%; display: table; position: static; overflow: hidden;"><div style="display: table-cell; vertical-align: middle; width:100%;">';
 9  code += '<img onclick="this.parentNode.parentNode.parentNode.removeChild( this.parentNode.parentNode); document.body.style.overflow = \'visible\';" ';
10  code += 'style="max-width:' + window.innerWidth + 'px; max-height: ' + (window.innerHeight - 20) + 'px;" src="' + url + '" /><br />';
11  code += 'click the photo to return to the blog. click <a href="' + url + '">here</a> to go to the image file. copyright &copy; by whoever.';
12  code += '</div></div>';
13  el.innerHTML = code; 
14  document.documentElement.appendChild(el);
15  return false;
16}
17//]]>
18</script>

let me explain what it does. lines 1, 2, 17 and 18 tell your browser that there’s a javascript coming up in an xhtml compatible way; if you are using plain html (instead of xhtml), you might want to change lines 2 and 17. now let’s continue with the main function, showPicture(), expecting one parameter called url.
first, the function disables scrolling for the main document (line 4), as otherwise the user can scroll around while viewing the picture (which won’t move). then it creates a new <div> tag (line 5) and eventually adds it as a child to the document root (line 14). the tag is equiped with a css style (line 6) telling the browser to display it at a position relative to the browser’s viewpoint, namely in the upper right corner with the size equal to the viewpoint size – meaning it covers the whole viewpoint. it is colored in black, with text in it being centered and written in white. then, in line 7, a onclick event is set for the tag. it simply waits until the tag is clicked (or anything inside that tag, to be precise) and removes the tag, which closes the “fullscreen” view. it also enables scrolling of the document (which was disabled in line 4). the lines 8 to 12 build up the html code which will be put into the <div> tag in line 13. in line 8, we create two <div> tags which will ensure a vertical centering in sane browsers. lines 9 and 10 inserts the image specified in url; here, i set the max-width and max-heigth parameters to essentially the viewport size, which ensures that the image will be rescaled to fit into the viewpoint. i substract a bit on the vertical size so there’s some space for the “disclaimer”. the “disclaimer” is added in line 11, it tells the user how to get rid of the fullscreen view, provides a link to the image file, and shows a copyright notice.
you can adjust the code as you like, for example you can change the colors and the disclaimer. also adding keyboard support might be nice, so you can get rid of the viewer when pressing escape. i guess it isn’t too hard to do, i’ll probably do that somewhen later. you can also move all the formatting into your .css file, by replacing the el.style.cssText = “…”; by el.className = “imageviewer”; or however you want to call the css class for that <div>.
posted in: www
tags:
places: