Though I’ve been using emacs to for quite a while, and often write plain text documentation in it, I’ve never previously explored org-mode. Recently however, I had some table heavy documentation to write and export to HTML and PDF, and decided to finally learn org-mode. Exporting to a nice HTML document was not too much trouble, there are many fine CSS layouts on github to pick from. I ended up basing my format off of orgcss with a few small modifications.

However when I tried to export to PDF, I hit a bit of a problem as Org Mode exports to PDF by first exporting to LaTeX, then then on to PDF. I had to install about a gigabyte of packages on my OS in order to get this working, and when I did, I was rewarded with a document that looked pretty much like any computer science thesis or research paper from the past 40 years. LaTeX is of course, highly configurable, and I could tweak the visual output to be more to my liking, but I decided to go a different route. I figured since I already had HTML that I liked, why couldn’t I just go Org -> HTML -> PDF, instead of Org -> LaTeX -> PDF.

I first tested this out by opening my HTML in chromium and just doing ‘print PDF’. I was happy enough with the result, but didn’t want to have to do this manually each time I wanted a PDF, so I set out to automate the process. My goal was a shell script could run on the command line that takes an Org file as input and outputs a PDF version in a single step. In addition, I wanted to be able to run this script right from an emacs buffer and export a PDF right from the buffer contents (without needing to save first). I also decided to add a flag to automatically pop the result up in a viewer for easy iterating while writing org files. I just M-x org-to-pdf and within a couple of seconds (longer for longer org files), I’m greeted by the rendered output of my work.

The script relies on emacs to do the Org->HTML conversion, and calls chromium to do the HTML->PDF conversion. Viewing of a PDF is handled by evince. The rest is just glue and error handling:

#!/bin/sh

display_help()
{
  echo "Usage: 2pdf.sh  [-v] [-k] <input file>" >&2
  echo "   <input file> can be .html or .org" >&2  
  echo "   -v - Open PDF in evince after creation. \
           (will first kill any open evince with the same file displayed" >&2  
  echo "   -k - Keep interim .html file when converting .org to .pdf"
  echo "   -h - Show this message" >&2
  echo  >&2
  echo "   PDF file will have same basename as input file \
           and .pdf extension"  >&2  
  echo "   Will read .org input from stdin if passed - as an input file. \
           In this case output file will be 2pdf.tmp.pdf" >&2  
  exit 0;
}

while getopts hvk opt
do
    case $opt in
        v)    viewPdf=1
              ;;
        k)    keepHtml=1
              ;;
        h)    display_help
              ;;
    esac
done
shift $(($OPTIND - 1))

if [ "$#" -ne 1 ]; then
  display_help
fi

if [ "$1" == "-" ]; then
    filename=2pdf.tmp.org
    cat > $filename
else
    filename=$1
fi

if ! [ -e "$filename" ]; then
  echo "$filename not found" >&2
  exit 1
fi

BASE=`echo "${filename%.*}"`
EXT=${filename##*\.}
PDF=${BASE}.pdf
HTML=${BASE}.html


case "$EXT" in
    html) 
	  HTML=$filename
          keepHtml=1
          ;;
    org)  emacs --batch --eval "(require 'org)" "$filename" \
          --funcall org-html-export-to-html || exit 1
	  ;;
    *)    echo " $filename : Unsupported Inputfile Type" >&2
          exit 1
          ;;
esac

chrome --disable-gpu --headless --print-to-pdf-no-header \
       --print-to-pdf="$PDF" "$HTML" || exit 2

if [ ! -z "$viewPdf" ]; then
  pkill -f "evince $PDF"
  nohup evince "$PDF"  >/dev/null 2>&1 &
fi

if [ -z "$keepHtml" ]; then
   rm "$HTML"
fi

With this I can call 2pdf.sh orgfile.org to generate orgfile.pdf. If I add a -k onto the command line, I also get an orgfile.html, and a -v will open up the PDF viewer after creating the PDF. I also added a pkill to kill any existing viewers already showing the same file, so I didn’t have to manually close them or end up with dozens when iterating. The script can also take org file input on stdin like cat orgfile.org | 2pdf.sh - . Finally if passed an HTML file instead of a org file, it only performs the HTML->PDF conversion (only .org format is supported on stdin though)

Creating an emacs function to call this was as simple as adding a few lines to my .emacs. I don’t speak much lisp, so I based this off of a recipe from “Mastering Emacs”.

(defun org-to-pdf ()
  "Converts org buffer contents to pdf"
  (interactive)
  (shell-command-on-region
   (point-min)
   (point-max)
   "~/scripts/2pdf.sh -v -"
   nil
   nil
   "*PDF Conversion Error Buffer*"
   nil))

Its little weird that I’m calling a shell script from emacs which then itself runs a headless emacs to do it’s work. I wrote the script so it can also take html as its input and just do the PDF conversion. At some point I should learn enough elisp to call org-html-export-to-html, extract the filename from the current buffer, replace the .org with .html, and then call 2pdf.sh on that, but this method works just fine for now.

One last little annoyance, which is always a problem when creating PDFs from HTML, is page breaks in dumb places, like right after a heading, or after the first line of a table. I’ve never found a good automated solution to this, but for Org Mode exports, I just manually put in html page breaks before the dumb place it would otherwise break. This means that if I further expand the document, I’d need to move my manual page breaks, but at least its a workaround. I put a macro in my Org files:

#+MACRO: new_page @@html:<p style="page-break-before: always">@@

I can then insert a manual page break with {{{new_page}}} .

Ever since I started using Linux and Unix systems in the mid-90’s, I’ve always relied on either XTerm or urxvt for my terminal needs while in X. These have always worked well, and provided me with exactly what I want when I want a terminal– a black box where I can type in and view text– nothing else taking up space or demanding my attention. No menubars or toolbars, no scrollbars, nothing but text. Currently I’m using xterm, with mostly default settings. As far as visuals, my only custom configuration is:

xterm*background: black
xterm*foreground: lightgray
xterm*faceName: monospace:pixelsize=12

Since I’ve been using the Xfce variant of GhostBSD, my computer came with xfce4-terminal pre-installed for me. However, with all the extra widgets visible and the inexplicably huge default text size, it never really occurred to me that this was a terminal to get real work done in beyond running sudo pkg install xterm in it. However, just because it has bad defaults, doesn’t mean its a bad terminal. The other day I actually opened it to check if some bad behavior in XTerm was also present there (It was). Once I had it open though, I decided to try to tame it.

The first few steps are straight forward. In the “General” Tab of the Preferences Dialog, set the Scrollbar to “Disabled”, then on the “Apperance” Tab, I set the Font to Monospace Regular with a size of 9. I’m not sure why size 9 on xfce4-terminal matches size 12 on XTerm, but it does. While on that tab, disable displaying of the menubar and toolbar. Next switch over to the “Colors” tab and select the helpfully provided “XTerm” palette under ‘Presets’. After this swap the text color and background color to get the classic one-true-terminal-colorscheme of light grey text on a black background. Finally, on the “Advanced” tab, make sure the default character encoding is set to UTF-8.

After restarting xfce4-terminal, it looks a lot like Xterm now at first glance, but we aren’t quite there. This is painfully obvious upon running emacs and getting a completely different color scheme than on the same system under Xterm. This can be resolved with some environment variable tweaks:

unset COLORTERM
export TERM=xterm

After this, theres still one more problem though that took me a bit of googling. Under XTerm, the Emacs minibuffer prompt was aqua colored and easy to see on the black background. On xfce4-terminal, however, it was drawing as a low-contrast dark blue and was mostly unreadable. While I could tweak the color on the ‘Colors’ Tab of xfce4-terminal, this would also change the color of other blue elements, like the blue hackground color of highlighted text, which is supposed to be blue. It turns out that Emacs understands the idea of XTerm with a black background and automatically compensates when picking colors, but for xfce4-terminal its trying to pick a color that would work well on a white background. Throwing a (setq frame-background-mode 'dark) into my .emacs took care of this.

So that did’t take too much work, but it was a fair amount of work just to get back to the starting point, so why bother with xfce4-terminal at all if you’re just going to make it look exactly like XTerm? Well in my case, one thing I wanted to try adding to my workflow was to use a tabbed terminal. I’ve tried a few times running XTerm inside of tabbed but it never took with me, maybe it will be different with the natively supported tabs in xfce-terminal

I usually have one big xterm on my main workspace ssh’d into work running GNU Screen with a bunch of windows logged into a bunch of machines there. Whenever I need to do something on my local machine, however, I just pop open a new XTerm, and often end up with marathon XTerm closing sessions every few days when it gets completely out of hand. I’m hoping that by having a “remote” tab and a “local” tab on my main work terminal with an easy hotkey to switch back and forth, I can train myself to use that in cases where I’d normally just spawn a new XTerm real quick, and then forget to close it when I’m done.

Client Side Decorations, where the application draws its own window decorations instead of the Window Manager, is just a terrible idea. Why go through the trouble of picking out a window manager you like, configuring and theming it exactly how you like it, only to have some random application decide it knows better than you how you want your windows to act. Until recently the only application I used that insists on client side decorations is my PDF viewer, evince. Unlike other windows, it doesn’t get an OpenBox title bar, so not only does it look out of place on my desktop, but I don’t have my normal set of OpenBox right-click-menu options. Honestly I need to open a PDF so infrequently that the annoyance was just below my threshold of finding a different application for PDFs or finding a solution.

But then recently, GhostBSD popped up a notification that it had new updates for me, which included an upgrade to Xfce 4.16. Unfortunately the Xfce developers decided to force client side decorations on all their users in this version, without even an option to disable them and re-enable sanity. I didn’t want to forgo the upgrades, so I finally spent the time finding a solution for disabling CSD system wide.

Client side decorations are a “feature” of gtk-3.0, and have been used in Gnome for some time, so some kind soul created gtk3-nocsd, which provides a replacement library to LD_PRELOAD to disable client side decorations and works on many Linux distributions. While there is no port / package for FreeBSD, its quite easy to download and build from source.

First, download the source, from this Github repro. There are a few different copies of it available on Github, but this one has the necessary compile fixes for FreeBSD already merged. The instructions provided in the README pretty much work on FreeBSD, there are only a couple of tricks:

First, the package names it lists as pre-requisites don’t exist under those names on FreeBSD. You won’t need any of the ‘devel’ packages, since the normal gtk3 packages include the necessary header files. If you already have Xfce or similar environment running, you probably already have all the gtk development files you’ll need. On my GhostBSD system, they were already present. There’s no pkg-config package on FreeBSD, that is provided by the pkgconf package, but again, I already had that installed. The only package I needed to install was gmake since the tools relies on GNU Make to build.

Second, download and extract the source from Github, enter the directory, and type gmake. It spit out a few warnings for me, but did successfully build. It creates both a binary and shared library. To test that it works, run a program using the binary as such:

./gtk3-nocsd /usr/local/bin/evince

If all is well, the program will pop up with normal window decorations. To make this the new default system wide, put the shared library somewhere and add these lines:

export GTK_CSD=0
export LD_PRELOAD=/path/to/libgtk3-nocsd.so.0

To somewhere where they will always be set when X applicatons run. In GhostBSD I added them to my .xprofile, but depending on how you start X, you might need to put them somewhere different. Important thing is to have those environment variables set whenever you launch a program.

Quick and easy, and sanity is restored.

I wrote about a method I had used to handle Manual X11 Clipboard Paste Selection. There I set up a shell script hot key so that Shift-Ctrl-V would paste my PRIMARY X Selection (and Ctrl-V would paste my CLIPBOARD selection as normal. This worked (mostly), but it was a bit of a hack that I never really liked. I knew of an alternative, to configure XTerm to always copy to the CLIPBOARD instead of the PRIMARY selection, but I wasn’t enamored with that approach either, as I often want to use the PRIMARY selection.

I recently ran across a blog post XTerm: It’s Better Than You Thought which clued me into a much better way to handle this. in .Xresources, they add:

XTerm.vt100.translations: #override \n\
   Ctrl Shift <Key>C: copy-selection(CLIPBOARD) \n\
   Ctrl Shift <Key>V: insert-selection(CLIPBOARD)

This allows Shift-Ctrl-C and Shift-Ctrl-V to copy and paste to the CLIPBOARD selection, while still leaving the normal XTerm PRIMARY selection functionality alone.

More interestingly, however, that same blog introduced me to an easy way to open URLs in an XTerm in a webbrowser. They provide a hotkey and a script to cause XTerm to open a menu of all the URLs visible in the XTerm, and upon selecting one, open it in a browser. This is useful to me, since I always have an IRC Window open for chat with my co-workers, and links are often pasted into chat. Normally I have to select the link with the mouse, and then paste it into Chromium with middle click, like some sort of caveman. The .Xresources configuration worked fine for me, but I had a hell of a time modifying the shell script provided to work for me on FreeBSD.

The script as provided on the blog was

#!/bin/sh -e
grep -Eo '\bhttps?://\S+\b' |
    uniq |
    ifne rofi -dmenu -i -p "Open URL" -auto-select |
    xargs xdg-open

I swapped out the rofi -dmenu for just normal dmenu for aesthetic reasons, and was pretty quickly able to change the regex to 'https?://[^[:space:]]+[[:>:]]' which is the same functionality but works on BSD Grep. However even after that, while the script worked fine when run manually, when run from the hotkey inside XTerm, it would hang forever trying to read from stdin. After trying many, many, things, it turned out that, for whatever reason, XTerm wasn’t closing stdin after writing the screen out to it, and nothing I could do would convince it to do otherwise, so the script was always waiting on more input.

I did finally work out the following hack:

#!/usr/local/bin/zsh

screen=""
while read -t 0.1 line
do
    screen="$screen\n$line"
done
echo "$screen" | grep -Eo 'https?://[^[:space:]]+[[:>:]]'  |
    uniq |
    ifne dmenu -i -p "Open URL" |
    xargs xdg-open

Since cat would hang forever until stdin was closed, I had to find a way to read from stdin without waiting. read allows for a timeout value, but it only reads a line at a time, and strips newlines. Running it in a loop and concatenating all the lines together while reinserting the newlines does the trick. This means my script requires a shell like zsh or bash to run, since sh doesn’t support read, but that’s not a problem for me.

After these changes, the script worked great for me, until I tried to use it on a URL that line wrapped, and it cut off the URL at the newline. I can resolve this by not inserting the \n between each line, but then URLs that should end at the end of a line, get merged with the beginning of the next line. Fortunately, I really only want this script for pulling urls from irssi, so I should be able to remove the \n and write the regex to end at the next real line by pattern matching the date/username that starts each line on IRC. I haven’t implemented this yet, but it should take care of the issue.

Its unfortunate that XTerm doesn’t distinguish between actual newlines on the screen and linewraps. Dragging the XTerm big enough on so the URL doesn’t wrap, makes it detect the entire thing. I did try out the xfce4-terminal with its “Right Click and select Open-Url from the Menu” functionality, and it suffers from the same issue– it considers the URL to end at the line wrap as well.

I failed at NetBSD.

At least my first attempt to use it as a work environment went somewhere between fairly bad, and quite bad. My plan, as I wrote in the previous post was to leave my desktop booted into NetBSD for the full work day to use for everything except email and skype, which I’d access via my FreeBSD laptop sitting next to my workstation.

I made it about two hours.

Things started off alright, I started the work day by VPNing into my work network with openconnect and opening a couple of xterms ssh’d into my GNU Screen session for IRC and emacs. Then I opened a Firefox and started browsing through my JIRA tickets.

The Firefox on NetBSD self identifies as ‘Nightly’, so I guess it was built from a nightly build of Firefox instead of a release, and at least for me, its pretty glitchy. I often get partial page redraws, with random data from previous pages left behind on part of the page and other artifacts. Also sometimes it will completely refuse to go to a new URL and I have to quit and restart. I was considering maybe trying to install the Firefox ESR package instead, but then I stepped away from my computer for about twenty minutes while I was on a call.

When I came back, I clicked my panel launcher to open an xterm and nothing happened. Hmm. I also couldn’t launch a new Firefox. Click the button and nothing happens. Luckily I already had an xterm open on another workspace, and that seemed to be working so I tried running ‘xterm’ from there, at least now I could see the error message:

invalid mit-magic-cookie-1 keyxterm: xt error: can't open display: :0.0

I vaguely remember some stuff about mit magic cookies and .XAuthority from way back, so I decided to google the error to refresh my memory. Immediately Firefox (Nightly), told me it had upgraded in the background and needed to restart, then quit, and of course when it tried to restart failed. So I googled the solution on my FreeBSD laptop instead.

I found many similar error reports with various fixes, but none of them seemed to apply for me, and none of the obvious stuff fixed it. I’m sure there’s an easy fix and I was just overlooking something, but I was on the clock and had other things I needed to work on, so I called my experiment a failure and booted back into GhostBSD.

Most of the similar mit magic cookie issues I see involve accessing X remotely, and its pretty weird to see it on a local connection and to have it just happen suddenly when previously the logged in session was able to create X windows just fine.

It will have to be a mystery for another time though when I have some free time to play around more with NetBSD.