Archive

Archive for the ‘best practices’ Category

Using GPG for Public-Key Cryptography

September 25, 2016 Leave a comment

Wikipedia has an accessible introduction of the subject. The GNU Privacy Manual is a good resource for detailed documentation.

Export Public Keys

To allow us to encrypt files with the public key of our correspondent, we first need to have access to their public key. Users may export their public key as follows:

% gpg --export --armor musa@persia.com \
      \<musa@persia.com\>.public.gpg-key

Import Public Keys

We may import public keys into our public keyring as follows. Doing so will enable us to encrypt data using this key.

% gpg --import \<musa@persia.com\>.public.gpg-key

Delete Public Keys

We may remove the public key from our public-key keyring by using name.

% gpg --delete-key musa@persia.com

List Public Keys

We can see the public keys listed in our public-key keyring using --list-keys

% gpg --list-keys
pub   4096R/9E520AE7 2016-09-15
uid                  Umar Kalim <umar@vt.edu>
sub   4096R/38C5C69A 2016-09-15
sub   4096R/5DE83AAF 2016-09-15

pub   4096R/B7AC8904 2016-09-25
uid                  Musa Al-Khwarizmi <musa@persia.com>
sub   4096R/4E74D6FE 2016-09-25
sub   4096R/7756B7E1 2016-09-25

Encrypting Files

To encrypt data we need to use the recipient’s public key. We may identify the recipient using the -r or the --recipient option followed by the recipients name or email. The --encrypt option highlights our intent to encrypt. By default gpg outputs encrypted content in binary format. To generate output in ASCII-armor format, we use the --armor option.

% gpg --encrypt --armor -r musa@persia.com file-1.pdf

Signing and Encrypting Files

By signing the contents, the recipient can guarantee that the contents were signed by our primary key.

% gpg --encrypt --sign --armor -r musa@persia.com file-1.pdf

Decrypting Files

Assuming that the private key associated with the public key used to encrypt the data is in our keyring, we may decrypt the encrypted contents as follows.

% gpg --decrypt file-1.pdf.asc > output.pdf

Creating a GPG Keypair for Public-Key Encryption

September 25, 2016 Leave a comment

GPG may be used to encrypt data and protect it from prying eyes. To use the public-key encryption method, we first need to generate a private-public keypair. This post summarizes the process of creating a new private-public keypair.

Wikipedia has an accessible introduction of the subject. The GNU Privacy Manual is a good resource for detailed documentation.

Choosing a Passphrase

When prompted for a passphrase, please follow guidelines for a strong passphrase. See Stanford University’s guidelines for choosing a strong passphrase.

Generating the Keypair

When generating the keys, it is recommended that we use the highest possible values for key lengths. This is relevant because, with the evolution of compute capabilities and accelerators (e.g., GPUs) it is becoming easy by the day to break keys with shorter lengths. Using longer keys would make it much more computationally challenging to break them.

It is also a good practice to set an expiry date for the keys. For the sake of simplicity, we’ll not set an expiry date in this post.

% gpg --gen-key
gpg (GnuPG) 1.4.18; Copyright (C) 2014 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y

You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"

Real name: Musa Al-Khwarizmi
Email address: musa@persia.com
Comment:
You selected this USER-ID:
    "Musa Al-Khwarizmi <musa@persia.com>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.    

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
...........+++++
.......+++++
gpg: key B7AC8904 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   3  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 3u
gpg: next trustdb check due at 2018-08-19
pub   4096R/B7AC8904 2016-09-25
      Key fingerprint = E79A 6B36 C40A AE5D 7E0B  3EB6 2108 BEBE B7AC 8904
uid                  Musa Al-Khwarizmi <musa@persia.com>
sub   4096R/4E74D6FE 2016-09-25

Strengthening Hash Preferences

It is preferable to use a stronger hashes. We may change the preferences using the --edit-keys option. The options used below are setpref SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed.

% gpg --edit-key musa@persia.com
gpg (GnuPG) 1.4.18; Copyright (C) 2014 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub  4096R/B7AC8904  created: 2016-09-25  expires: never       usage: SC
                     trust: ultimate      validity: ultimate
sub  4096R/4E74D6FE  created: 2016-09-25  expires: never       usage: E
[ultimate] (1). Musa Al-Khwarizmi <musa@persia.com>

gpg> setpref SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed
Set preference list to:
     Cipher: AES256, AES192, AES, CAST5, 3DES
     Digest: SHA512, SHA384, SHA256, SHA224, SHA1
     Compression: ZLIB, BZIP2, ZIP, Uncompressed
     Features: MDC, Keyserver no-modify
Really update the preferences? (y/N) y

You need a passphrase to unlock the secret key for
user: "Musa Al-Khwarizmi <musa@persia.com>"
4096-bit RSA key, ID B7AC8904, created 2016-09-25

pub  4096R/B7AC8904  created: 2016-09-25  expires: never       usage: SC
                     trust: ultimate      validity: ultimate
sub  4096R/4E74D6FE  created: 2016-09-25  expires: never       usage: E
[ultimate] (1). Musa Al-Khwarizmi <musa@persia.com>

gpg> save

Revocation Certificate

In case the primary key is compromised, we may use the revocation certificate to inform people about the key being stolen and not to trust content encrypted or signed using those keys. Please remember to keep this certificate in a different place than the master keypair.

% gpg --output \<musa@persia.com\>.gpg-revocation-certificate \
      --gen-revoke musa@persia.com

Exporting Primary and Public Keys

The commands below will generate the primary and public keys.

% gpg --export-secret-keys --armor musa@persia.com \
      \<musa@persia.com\>.private.gpg-key
% gpg --export --armor musa@persia.com \
      \<musa@persia.com\>.public.gpg-key

Adding Signing Subkey

Creating subkeys allows us to remove the master keys from the keyring and dedicated subkeys for specific purposes. If we happen to lose our subkeys, we may easily revoke those subkeys with the master keypair. Note that the revocation certificate is meant for the master keypair and not the subkeys. In the example below we create a signing subkey.

% gpg --edit-key musa@persia.com
gpg (GnuPG) 1.4.18; Copyright (C) 2014 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub  4096R/B7AC8904  created: 2016-09-25  expires: never       usage: SC
                     trust: ultimate      validity: ultimate
sub  4096R/4E74D6FE  created: 2016-09-25  expires: never       usage: E
[ultimate] (1). Musa Al-Khwarizmi <musa@persia.com>

gpg> addkey
Key is protected.

You need a passphrase to unlock the secret key for
user: "Musa Al-Khwarizmi <musa@persia.com>"
4096-bit RSA key, ID B7AC8904, created 2016-09-25

Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
......+++++
.+++++

pub  4096R/B7AC8904  created: 2016-09-25  expires: never       usage: SC
                     trust: ultimate      validity: ultimate
sub  4096R/4E74D6FE  created: 2016-09-25  expires: never       usage: E
sub  4096R/7756B7E1  created: 2016-09-25  expires: never       usage: S
[ultimate] (1). Musa Al-Khwarizmi <musa@persia.com>

gpg> save

Using GPG for Symmetric Cryptography

September 25, 2016 Leave a comment

GPG may be used to encrypt data and protect it from prying eyes. We may either use symmetric-ciphers or the public-key cryptography method to protect our data. In this blog, I’ll summarize the use of symmetric ciphers on OSX.

When we have more than one file to encrypt, it would be prudent to encrypt an archive of the files instead of encrypting them one by one.

% ls -la
-rw-r--r--@   1 umar  staff   916K Sep 25 01:08 file-1.pdf
-rw-r--r--@   1 umar  staff   5.3M Sep 25 01:08 file-2.pdf

% tar cfvj my-files.tar.bz2 file-*
a file-1.pdf
a file-2.pdf

Choosing a Passphrase/Password

When encrypting files, GPG would prompt for a passphrase. Please follow guidelines for a strong passphrase. See Stanford University’s guidelines for choosing a strong passphrase.

Encrypting a File

The -c or --symmetric option is used to encrypt files with a symmetric cipher.

The output files may be saved in binary or ASCII-armor format. The ASCII-armor versions, although relatively larger in size, are typically used when sharing encrypted files through email. The default option is to generate binary output (i.e., .gpg files). The --armor option generates the ASCII-armor version of the encrypted output.

GPG uses CAST5 as the default cipher. However, the 256-bit Advanced Encryption Standard (i.e., AES256) is one of the ciphers recommended by the Computer Security Resource Center at NIST. We can select the encryption algorithm with the option --cipher-algo.

% # Binary output
% gpg --output my-files.tar.bz2.gpg \
      --cipher-algo AES256 \
      --symmetric \
      my-files.tar.bz2

% # ASCII-armor output
% gpg --output my-files.tar.bz2.gpg \
      --cipher-algo AES256 \
      --armor \
      --symmetric \
      my-files.tar.bz2

% ls -la
-rw-r--r--@   1 umar  staff   916K Sep 25 01:08 file-1.pdf
-rw-r--r--@   1 umar  staff   5.3M Sep 25 01:08 file-2.pdf
-rw-r--r--    1 umar  staff   4.7M Sep 25 01:13 my-files.tar.bz2
-rw-r--r--    1 umar  staff   6.4M Sep 25 01:17 my-files.tar.bz2.asc
-rw-r--r--    1 umar  staff   4.7M Sep 25 01:16 my-files.tar.bz2.gpg

Decrypting a File

The -d or --decrypt option may be used to decrypt the encrypted files (whether .asc or .gpg)

% gpg --output decrypted-files.tar.bz2 \
      --decrypt \
      my-files.tar.bz2.asc
gpg: AES256 encrypted data
gpg: encrypted with 1 passphrase

% ls -la
-rw-r--r--    1 umar  staff   4.7M Sep 25 01:28 decrypted-files.tar.bz2
-rw-r--r--@   1 umar  staff   916K Sep 25 01:08 file-1.pdf
-rw-r--r--@   1 umar  staff   5.3M Sep 25 01:08 file-2.pdf
-rw-r--r--    1 umar  staff   4.7M Sep 25 01:13 my-files.tar.bz2
-rw-r--r--    1 umar  staff   6.4M Sep 25 01:17 my-files.tar.bz2.asc
-rw-r--r--    1 umar  staff   4.7M Sep 25 01:16 my-files.tar.bz2.gpg

We can confirm that the output after decryption is the same as the input before encryption using MD5 checksums.

% md5sum my-files.tar.bz2 decrypted-files.tar.bz2
MD5 (my-files.tar.bz2)        = cbdff2d18d943d36c4224aada805e6ce
MD5 (decrypted-files.tar.bz2) = cbdff2d18d943d36c4224aada805e6ce

Embed Fonts in PDFs

February 3, 2015 Leave a comment

It is typical of IEEE/ACM conferences to ask for pdf files with embedded fonts. When the submission deadline is fast approaching, it is an unpleasant experience to see a prompt saying that the submission does not meet the requirement of embedded fonts.

Until 2014, pdflatex (as part of MacTeX) was configured not to embed fonts by default. There is a simple solution to this problem. Run the following commands and any pdf generated with pdflatex will have fonts embed in them.

$ updmap --setoption pdftexDownloadBase14 true
$ updmap

pdffonts is a nifty tool, which can be used to list the fonts embedded in a pdf document. Below is a log of the outputs before and after I changed the configuration.

$ pdffonts test.pdf                                                                                                        
name                                 type              emb sub uni object ID
------------------------------------ ----------------- --- --- --- ---------
PQJFKD+CMR17                         Type 1            yes yes no       4  0
OIKBLN+CMR12                         Type 1            yes yes no       5  0
VDNHSL+CMR10                         Type 1            yes yes no       6  0
Times-Roman                          Type 1            no  no  no       7  0
Helvetica                            Type 1            no  no  no       8  0
Courier                              Type 1            no  no  no       9  0
IAPCTB+URWPalladioL-Roma             Type 1            yes yes no      10  0
IQSCJF+URWChanceryL-MediItal         Type 1            yes yes no      11  0
ZINFER+CenturySchL-Roma              Type 1            yes yes no      12  0
SXEYYX+URWBookmanL-Ligh              Type 1            yes yes no      13  0

$ updmap --setoption pdftexDownloadBase14 true
$ updmap
...

$ pdffonts test.pdf                                                                                                        
name                                 type              emb sub uni object ID
------------------------------------ ----------------- --- --- --- ---------
PQJFKD+CMR17                         Type 1            yes yes no       4  0
OIKBLN+CMR12                         Type 1            yes yes no       5  0
VDNHSL+CMR10                         Type 1            yes yes no       6  0
RFZQJL+NimbusRomNo9L-Regu            Type 1            yes yes no       7  0
KRXWNA+NimbusSanL-Regu               Type 1            yes yes no       8  0
IJPKSY+NimbusMonL-Regu               Type 1            yes yes no       9  0
IAPCTB+URWPalladioL-Roma             Type 1            yes yes no      10  0
IQSCJF+URWChanceryL-MediItal         Type 1            yes yes no      11  0
ZINFER+CenturySchL-Roma              Type 1            yes yes no      12  0
SXEYYX+URWBookmanL-Ligh              Type 1            yes yes no      13  0

We can see that all the fonts are embedded in the pdf.

pdffonts is available as part of xpdf. It is available as a Homebrew recipe and can be installed using the following command:

$ brew install xpdf
...

An alternate solution is to open the pdf file in Preview and export or print as a pdf. The resulting file will have embedded fonts.

If you have figures as eps files, then those files may not have embedded fonts as well. You can convert them to pdf while embedding fonts. (pdflatex will not embed fonts in figures that were created by other software.)

ps2pdf13 -dPDFSETTINGS=/prepress figure-without-fonts.eps figure-with-fonts.pdf
Categories: best practices Tags:

Using TMUX to Manage Workspace

Tmux makes management of your workspace easy, particularly when we use remote servers. I found tmux to be much more versatile than screen. Particularly the split windows are much more convenient to setup and manage. Below is a screen shot of a tmux session.

A screenshot of a TMUX session

A screenshot of a TMUX session

I started using screen when I had to code software and run experiments that would run overnight. Screen allowed me to detach the terminal from a session; I could setup and start an experiment, pack my bags, head home and later reconnect to the same session. The experiments would continue to run within the session. Had I not used screen, the sessions would have terminated when I exited the console.

Then I discovered tmux. What I like most about tmux is that I did not lose the split windows configuration when I detached from a session.

Below is a short tutorial about using tmux, which will get you started.

Tmux uses sessions, windows and panes to organize the workspace. As shown below, a session may include multiple windows, which in turn may include multiple panes. The screenshot above is an example of a window with four panes. Each pane can run an independent console instance. We can choose the number of windows we need for our workspace and number of panes within them. It is simple to configure the layout of the panes and their sizes.

Session, Windows and Panes

Sessions

To start a tmux session, we simply execute the command tmux. The session starts with one window and a pane within it. As with screen, tmux uses key combinations for shortcuts.

CTRL+b s shows all the sessions. We can do the same with tmux ls.

We can detach from a session with CTRL+b d. To reattach to the first available session we use tmux a. tmux a -t <session name> allows us to attach to a session named session name.

Windows

We can create new windows within the session with CTRL+b c. CTRL+b , renames the window. These names are visible in the status bar at the bottom of the console.

CTRL+b n and CTRL+b p moves to the next and previous windows respectively. CTRL+b w lists all windows along with their numbers. We can switch to particular window with CTRL+b <number>.

With CTRL+b & we can kill the current window.

Panes

By default a window has a single pane. To create new panes, we split planes horizontally or vertically. We can split a plane horizontally with CTRL+b % and vertically with CTRL+b ".

To move between panes we use CTRL+b { and CTRL+b } to go left or right respectively. We use CTRL+b o to switch to the next pane in the window.

CTRL+b q highlights each pane with its number.

We can rotate the consoles in the panes with CTRL+b CTRL+O

Tmux has default layouts for the panes. We can switch between those using CTRL+b space.

Commands

We can execute commands that are understood by tmux. CTRL+b : allows us to switch to command mode. For example, to resize a pane upward by 10 cells we use CTRL+b : resize-pane -U 10. The options for up, down, left and right are -U, -D, -L and -R.

Summary

If we are not sure about the key bindings, we can use CTRL+b ? to look them up. The man pages are always a good source of documentation.

Here is the list of all the key combinations I mentioned above:


tmux 		Start the session
tmux a		Attach to first available session
tmux a -t session-name
tmux ls 	List all sessions
CTRL+b  d	Detach client from current session
CTRL+b  s	List all sessions

CTRL+b  c	Create window
CTRL+b  ,	Rename window
CTRL+b  n	Next window
CTRL+b  p	Previous window
CTRL+b  w	List all windows
CTRL+b  <number> 
			Move to window number
CTRL+b  &	Kill the current window

CTRL+b  %	Split horizontally
CTRL+b  "	Split vertically
CTRL+b  q	Show pane numbers
CTRL+b  o	Next pane
CTRL+b  Ctrl-O	
			Rotate consoles within panes
CTRL+b  {	Jump to the left pane
CTRL+b  }	Jump to the right pane
CTRL+b  <space> toggle through the different layouts

CTRL+b  :	Enter command mode
CTRL+b  : resize-pane -U 10 
			Resize the pane upwards

CTRL+b  ?	Show key bindings

Categories: best practices Tags: ,

Stamp Papers with Their Citation

Wouldn’t it be convenient if the citation of the paper was available as part of the pdf document? You wouldn’t have to google for the information.

We can do so with the help of a simple perl script — the stamp is formatted using latex and is integrated with the original document to generate another version that has the stamp at the top of the page.

Below is a snapshot of document that was stamped using this script. Note the publication venue, location, month and year of publication at the top of the paper. The pdf is available here.

stamp-research-paper-example

Here is the script:

#!/usr/bin/perl

$numArgs = $#ARGV + 1;
if($numArgs != 3){
    print "Usage: ./stamp.pl <input pdf> <output pdf> \"year etc.\"\n";
    exit;
}

$input_pdf = $ARGV[0];
$output_pdf = $ARGV[1];

$text = <<END 
\\documentclass[letter]{article}
\\usepackage{fancyhdr}
\\usepackage[left=2cm,top=1cm,right=2cm,nohead,nofoot]{geometry}
\\pagestyle{fancy}
\\begin{document}
%\\fancyhead[CO,CE]{\\protect \\centering $ARGV[2]}
\\centering $ARGV[2] 
\\fancyfoot[c]{} 
\\end{document}
END
; 

open TEX_OUT, '>stamp.tex';
print TEX_OUT $text;
close TEX_OUT;

system('pdflatex stamp.tex');
system("pdftk $input_pdf stamp stamp.pdf output $output_pdf"); 
Categories: best practices

Using latexdiff to Highlight Revisions in a LaTeX Project

April 21, 2014 Leave a comment

Typically research papers are put together as LaTeX projects. I was under the impression that it is a challenge to keep a visual track of who made what changes; typically there are more than one authors on the paper and we try to learn from the experienced writers. But then I came across a useful tool, latexdiff, which opened the door for many possibilities.

What latexdiff does is that it highlights, in color, the differences between two LaTeX files. Consider a trivial example below. Latexdiff compares the two LaTeX files (original.tex and revised.tex) and generates a LaTeX file as output (diff.tex), which can then be compiled into pdf to view the output.

latexdiff original.tex revised.tex > diff.tex
pdflatex diff.tex

An example is shown below, where red color highlights the text that was deleted and blue color highlights the text that was added.

latexdiff example output

When collaborating, the LaTeX project includes multiple files that are pulled into the primary LaTeX file. This can be done using the flatten option.

latexdiff --flatten original.tex revised.tex > diff.tex

I’ll leave it to you to explore the documentation for other possibilities.

The tool can also be used along with version control systems (e.g., svn). Below is a crude bash script that pulls an old version from the svn repository and compares it to the version in your working directory.

#!/bin/bash 
# Version: 0.01
# Author:  Umar Kalim

if [ $# -ne 2 ] ; then
  echo "Usage: $0 <svn revision no.> <filename>"
  exit 1
fi

svn_revno=$1
filename=$2

mkdir tmp
echo "exporting svn repo $svn_revno"
svn export --force -r $svn_revno . tmp

echo "generating diff"
latexdiff --flatten tmp/$filename $filename > diff.tex

echo "compiling output with highlights"
pdflatex diff.tex

echo "cleanup"
rm -rf tmp
Categories: best practices Tags: