How To Run Jackknives¶

Getting Started¶

First, make a directory to do the jackknives in. I typically make these on cbassvisitor3 and give them names like v28allels_jk.

The jackknife code has lots of input options. These are primarily set in two/three files. The user must create the following files:

a dictionary (described below)
an options ini file (described below)
the master script that calls functions in the correct order, commonly called jk_run.py. (described below)

The user must also create the directories where the various outputs are to be saved. Typically these include

fileLists (make this sub-directory in your jackknife directory now)
gainFactors (make this sub-directory in your jackknife directory now)
maps (make this sub-directory in your jackknife directory now)
outputs (make this sub-directory in your jackknife directory now)
params (there is a directory on cbassvisitor3 that contains lots of example params files - you don’t need to make this directory)
tod (only make this if you want to copy your TOD here, you only might want to do this if you plan on making lots of cuts on the data)
webpages ((make this sub-directory in your jackknife directory now)

In the following sections we describe how to set up your directories, the scripts that can help you do that, and then finally how to set up the jackknives.

Dictionary File¶

The dictionary file tells the jackknife code where things live (e.g. TOD, maps, ini files for Descart, etc…). It is commonly called jk_dicts.py, but obviously you could call it what you want. Within your jk_run.py script you will want to:

import jk_dicts as jkd

An example dictionary can be generated using cbasspy.jackknives.jkexamples.generateDict(). Generate your example dictionary now.

The dictionary file defines the following dictionaries and functions

dirDict¶

A place to record the locations of relevant directories that the jackknife and mapmaking code will need.

todDir : Location of TOD .fits files
fileListDir : Location of file list .txt files for DESCART
mapDir : Where to save/find healpix maps
paramDir : Location of params.ini files for DESCART (you might want this to be the common params file directory on cbassvisitor3)

fileListDict¶

A place to record the file-list file-names that live in fileListDict and also give them a short name (the short names will be used to tell the jackknife wrapper which file list to use and also put in the output map file names, so don’t use spaces or funny characters!).

paramDict¶

A place to record the .ini files in paramDir and give them a short name (like the short names in fileListDict, these are used to tell the wrapper which ini files to use and puts this string in the output file names).

cutDict¶

Contains the instructions to give cbass.split_data to make the cuts on the TOD. There are two functions in the example jk_dicts.py.

printDict(dict)¶

A handy little function that prints the contents of “dict”

list()¶

Lists the dictionaries

Map objects¶

The jackknife code uses map objects, cbasspy.jackknives.jkmaps.CBASSmap. These objects have attributes containing all the information necessary to produce a map and methods to make it and read it in.

Within jk_run you will likely import jkmaps as:

import jkmaps as jkm

Maps have the following attributes

Name: map name has format [cutName]_[fileListName]_[paramName]_[coord]
cutName: name of cut on the data
cut: cbass.split_data command to obtain the cut
fileListName: short name of .txt file list
fileList: full file name
fileListDir: directory to find .txt file
paramName: short name of .ini parameter file
param: full parameter file name
paramDir: directory to find .ini file
coord: coordinate system of map, eg ‘c’, ‘g’
mapDir: where to find the map and cov .fits files
todDir: where to find the tod .fits files

To create a map object:

exampleMap = jkm.CBASSmap(cutName,fileListName,paramName,dir.coord)

where cutName, fileListName and paramName correspond to the names of items in the relevant dictionaries form jkd. dir is jkd.dirDict.

The built in methods allow you to

make the map, exampleMap.make_map() (Set to use 40 C-BASS core on glamdring).
run the split on those tod fits files, exampleMap.split_data() (Set to use the 40 C-BASS cores on glamdring).
read the map, IQUmap = exampleMap.read_map()
read the hits map, Hmap = exampleMap.read_hits()
read the covariance matrix, cov = exampleMap.read_cov()

You don’t have to use this just yet, we will use it shortly…

Note¶

You don’t want the cbass.split_data commands crashing into one another as it screws up the tod.fits files. The .split_data method just adds the command to the Glamdring queue, so to avoid things crashing into each other, only run one at a time!

Options File¶

The options.ini file contains the imput parameters used by the jackknife code. It sets the output options, what things to calculate etc… There are a lot of options to set!

Generate an example options file using cbasspy.jackknives.jkexamples.generateOptions().

Do this now, and edit any options that you want to. Make sure to make outDir and htmlDir!

Doing The Jackknives¶

The bulk of the jackknife code lives in cbasspy.jackknives.jkgeneral. Typically in jk_run you will import jkgeneral like:

import jkgeneral as jkg.

The main function in this module is cbasspy.jackknives.jkgeneral.jackknife().

The requried inputs are

maps: list of jkm.CBASSmap objects
JKtitle: name of jackknife
whichJK: list of pairs of maps to compare, eg [(0,1),(0,2),(1,2)]
jk_options: .ini file with jackknife options

There are also some short functions within this module that are used by the main jackknife function. The command you are probably interested in is:

jkg.jackknife(maps,JKtitle,whichJK,jk_options)

It runs the jackknives set out in whichJK comparing the maps in maps. Outputs have JKtitle in the name so it’s good to make this some thing both short and descriptive and unique! The outputs etc… are set in jk_options.

!!!At the moment you need to make the directories outDir and htmlDir that are set in jk_general.py!!!

Your jk_run.py Script¶

You want to run things in a sensible order. Set this in your own jk_run.py script. jk_run.py is then (unsuprisingly) the script that you call to do everyhting. Make your jk_run.py script now.

Typically jk_run.py will be laid out as follows:

# The capitalised word on the line below is used by jkg to identify
#  where the map info begins. Do not delete the line!
# START
import numpy as np
import healpy as h
import sys
import os
import time
import itertools
from cbasspy.jackknives import jkmaps as jkm
from cbasspy.jackknives import jkgeneral as jkg
import jk_dicts as jkd

Note the commented lines at the top, they are important. Include them! Copy the above lines of code into your jk_run.py script now.

Then make the map objects:

NIGHT_testMap = jkm.CBASSmap('NIGHT','testFiles','default_params',jkd.dirDict,'c')
DAY_testMap = jkm.CBASSmap('DAY','testFiles','default_params',jkd.dirDict,'c')

# END
# The capitalised word on the line above is used by jkg to identify
#  where the map info finishes. Do not delete the line!

Note the commented lines at the bottom, they are important. Include them! Make any map objects that you want to compare and make sure to copy the commented lines at the bottom - do this now!

Then run the cuts on the data and make the maps, you only have to do this once!:

NIGHT_testMap.split_data()
NIGHT_testMap.make_map()

DAY_testMap.split_data()
DAY_testMap.make_map()

You may not need to call split_data(), you will need to make the maps. Add these to your jk_run.py script now.

Finally run the jackknives:

maps = [jkm.NIGHT_testMap,jkm.DAY_testMap]
JKtitle = 'test'
whichJK = [(0,1)]
jk_options = 'jk_options.ini'

jkg.jackknife(maps, JKtitle, whichJK, jk_options)

Add calls to jkg.jackknife in your jk_run.py script now.

NOTE, wait until the maps have been made to run the jackknives. The jakknives can be run on the Glamdring queue using cbasspy.jackknives.jkgeneral.run_jackknife_glam().

It can be a good idea to make your jk_run.py script take some input options to only do what you want it to every time you call it. For example, the first time you run it you want it to just make the maps. The second time you want it to do the jackknives. (The zeroth time you might want it to apply a cut on the data).

Don’t call jk_run.py just yet, we have a little more to do!

Spice options¶

You need to specify the -optinfile used by PolSpice to create the power spectra. There are examples in /elephant/cbassvisitor3/spiceFiles/. The default is jk_spice.txt, this is probably the one you want to keep using.

Run the code¶

You can now run the jackknife code. Go for it!

Publishing To Web¶

The module cbasspy.jackknives.web_publish can be used to publish the results to the Oxford reduced data website.

Run like:

python web_publish.py JKtitle jk_options

It will also

copy the polSpice .txt files to output dir
change the permissions of all output files
replace the absolute file paths in .html file with relative ones
rsync over the outputs and html files to the web server

So far only tested on glamdring. Some hardwired commands will need editing on other machines.