In this tutorial, you will learn to:

• generate a HiPS from a collection of FITS images
• visualize the created HiPS in Aladin Desktop, Aladin Lite and ipyaladin
• extract FITS and PNG cutouts from the generated HiPS, with the HiPS2FITS cutout script
• (bonus) create a color HiPS, and extract color cutouts

Free disk space

10GB of free disk space is required to run this tutorial from end to end.

Tools

Download tools.tar.gz and uncompres it in the tutorial directory. This will create a directory tools containing:

• Hipsgen.jar: the software package dedicated to generate HiPS.
• Aladin Desktop: Aladin Sky atlas desktop application.
• hips2fits_cutout.py: the Python script creating FITS and PNG cutouts from a local HiPS.
• server_http_cors.py: a small utility to enable access to local filesystem through HTTP.

Java

A Java Runtime Environment is needed to execute Hipsgen and launch Aladin Desktop.

Python environment

You will also need a recent Python version (for the cutouts generation part of the tutorial). We suggest to create a new Python virtual environment. If you use conda, type:

conda create -n semiovfrance python=3.10

and activate the new environment with:

conda activate semiovfrance

Download requirements.txt and install the needed libraries:

pip install -r requirements.txt

Data

We will work from a collection of images from the HTTP (Hubble Tarantula Treasury Project: 30DOR). It is made up of 36 4216x5087 pixels FITS images, taken with the narrow filter F658N.
Download HTTP_F658N.tar.gz and uncompress in your working directory. It will create a new directory tree data/HTTP/F658N.

Hipsgen in 3 minutes: introduction to Hipsgen

Going further: Hipsgen full manual is available here.

First step: spatial index generation

Launch:

java -jar tools/Hipsgen.jar in=data/HTTP/F658N out=hips/F658N creator_did=test/P/HTTP/F658N INDEX

Parameter in specifies the location of the input images, out the location of the output (generated HiPS).

After running this command, you should see a new directory hips/F658N/HpxFinder. It provides with the list of images contributing to each tile (at the deepest order).

Example: here is the content of the file hips/F658N/HpxFinder/Norder14/Dir2168900000/Npix2168909825

{ "name": "jby01s010_drz", "path": "data/HTTP/F658N/jby01s010_drz.fits", "ra": "84.51993066121292", "dec": "-69.01307439161397", "cellmem": "85807516", "stc": "POLYGON J2000 84.44501200336873 -68.97586757860584 84.60792293438074 -68.97974489548126 84.59510328895387 -69.05024833550632 84.43167129497695 -69.04635856212786" }
{ "name": "jby01y010_drz", "path": "data/HTTP/F658N/jby01y010_drz.fits", "ra": "84.45327968761491", "dec": "-69.03507605998125", "cellmem": "85807516", "stc": "POLYGON J2000 84.37828568610226 -68.99786933460068 84.54135961681285 -69.00174641284528 84.52852820643011 -69.0722498781017 84.36493209047423 -69.06836032977662" }
{ "name": "jby01z010_drz", "path": "data/HTTP/F658N/jby01z010_drz.fits", "ra": "84.52353834020153", "dec": "-69.06206356030239", "cellmem": "85807516", "stc": "POLYGON J2000 84.44845170170487 -69.02485694193437 84.61172604832419 -69.02873372836892 84.59888016980786 -69.09923722451869 84.43508225651556 -69.09534795130955" }
{ "name": "jby03q010_drz", "path": "data/HTTP/F658N/jby03q010_drz.fits", "ra": "84.38744904070751", "dec": "-69.06570518314524", "cellmem": "85787168", "stc": "POLYGON J2000 84.46900860831003 -69.10116385481491 84.30486161165173 -69.10085914586092 84.30615307696785 -69.03020789312967 84.4697718289979 -69.03051162146811" }
            

These 4 images contribute to tile #2168909825 at order 14.

Second step: FITS tiles generation

Launch:

java -jar tools/Hipsgen.jar in=data/HTTP/F658N out=hips/F658N creator_did=test/P/HTTP/F658N TILES

At this step, your HiPS can be already visualized in Aladin Desktop or Aladin Lite.

Third step: PNG tiles generation

Not all HiPS clients support FITS tiles. It is a good idea to also generate PNG tiles which are smaller and more suitable for preview purpose.

Launch:

java -jar tools/Hipsgen.jar in=data/HTTP/F658N out=hips/F658N creator_did=test/P/HTTP/F658N pixelCut="0 5 log" PNG

The parameter pixelCut allows one to specify the minimal cut, the maximal cut and an optional stretch function. Pixels with values smaller than the minimal cut will turn out black whereas pixels with values greater than the maximal cut will turn out white in the PNG tiles.
If pixelCut is not specified, Aladin Lite will choose automatically cuts based on one sample image, which might or might not be illustrative of the whole images collection.

Lanch Aladin Desktop with:

java -jar tools/Aladin.jar

and drag'n'drop the HiPS directory hips/F658N to open it in Aladin Desktop.

Open the Pixel histogram and contrast controller for the newly created plane. From this window, you can switch between preview (PNG or JPEG) and FITS tiles with the whole dynamic.

Because of browser security restrictions, Aladin Lite can not access the local filesystem. Thus, we will serve the generated HiPS from a HTTP server running locally.
In a new terminal, cd to the tutorial directory and type:

./tools/server_http_cors.py

Files from the directory are now served through http://localhost:2024/.

Access the document located at http://localhost:2024/hips/F658N/.
It has been built by Hipsgen. It is based solely on the content of the properties file.
Edit the properties (located at hips/F658N/properties) by updating for instance the property obs_title. Save your change and reload the page http://localhost:2024/hips/F658N/ ; check that the title reflects your edit.

The HiPS landing page can be customized: let's add a targets list selector. Open the file hips/F658N/index.html in your favorite text editor, go to the bottom of the page and change the following line:

buildLandingPage();

Replace it with:

buildLandingPage({targets: ['HTR 13', 'Dor IRS 108', 'VFTS 442', '[BW2019] 30 Dor-10 6', 'RMC 136', 'NGC 2070', 'NGC 2060', 'RMC 140']});

Reload the landing page in your browser, and jump to a few targets from the selector.

ipyaladin is a Python widget enabling usage of Aladin Lite in Jupyter (notebooks, lab) environment.

Download the notebook ipyaladin-demo.ipynb, and run:

jupyter notebook ipyaladin-demo.ipynb

Execute the different cells.

Let's query VizieR catalogue J/MNRAS/389/678 through astroquery and overlay the result tables in ipyaladin. Add the following code in the notebook, and execute it:

import astropy.units as u
from astroquery.vizier import Vizier
Vizier.ROW_LIMIT = -1

result = Vizier.query_region("tarantula", radius=0.5*u.deg, catalog='J/MNRAS/389/678')

for t in result:
    aladin.add_table(t)
            

Clicking on the sources in Aladin Lite will display the corresponding line of the catalogue.

The HiPS2FITS service enables generation of FITS images cutouts of arbitrary size and resolution from HiPS datasets available in the HiPS network or publicly available through HTTP.

The HiPS we just created is not publicly available, hence HiPS2FITS service is not an option. But we can use the hips2fits_cutout.py Python script, available in the tools directory.

This command-line script can be used to create a single cutout using the following syntax:

./hips2fits_cutout.py <ra-center> <dec-center> <fov (image size) in degrees> \
            <width> <height> \
            <local-hips-path> <name-generated-cutout-file> <format (fits, png, jpg)> 

Create a single cutout

Let's create first a 2000x2000 FITS cutout, giving an overview of our HiPS:

./tools/hips2fits_cutout.py 84.605651 -69.126215 0.16 2000 2000 hips/F658N cutout.fits fits

Drag&drop the resulting cutout in Aladin Desktop or Aladin Lite.
We can also create a PNG cutout with:

./tools/hips2fits_cutout.py 84.605651 -69.126215 0.16 2000 2000 hips/F658N cutout.png png

Create many cutouts from a list of positions

If we want to create hundreds of cutouts, it is unconvenient and inefficient to execute our script for each cutout. hips2fits_cutout.py has a dedicated option -l to generate cutouts from a list, formatted as a CSV file.

Launch:

./tools/generate-cutouts-params.py

The script has queried SIMBAD TAP service, requesting SIMBAD objects in the vicinity of Tarantula nebula, and whose object type is different from Star:

 SELECT TOP 100000 ra, dec, main_id, oid, otype, galdim_majaxis
  FROM basic
  WHERE CONTAINS(POINT('ICRS', ra, dec), CIRCLE('ICRS', 84.67567, -69.11625, 0.53917)) = 1
    AND otype != '*'
   ORDER BY nbref DESC

We then filter the result to keep only objects within the coverage (MOC) of our HiPS:

sc = SkyCoord(table['ra'], table['dec'], unit='deg')
hips_coverage = MOC.load('hips/F658N/Moc.fits')
mask = hips_coverage.contains_skycoords(sc)

Have a look at the generated parameters file cutout-params.csv

We are now all set to create our collection of cutouts.
Execute:

mkdir my-cutouts

./tools/hips2fits_cutout.py -l cutout-params.csv

Visualize a few cutouts in the my-cutouts directory.

We can also generate an HTML page to easily visualize our cutouts:

./tools/hips2fits_cutout.py -l cutout-params.csv -html gallery.html

Open gallery.html in your browser.

We will first need to create two additional HiPS datasets, from filter F555W and filter F775W.
Download and uncompress HTTP F555W and HTTP F775W images.

Generate HiPS FITS tiles for F555W:

java -jar tools/Hipsgen.jar in=data/HTTP/F555W out=... creator_did=... INDEX

java -jar tools/Hipsgen.jar in=data/HTTP/F555W out=... creator_did=... TILES

Generate HiPS FITS tiles for F775W:

java -jar tools/Hipsgen.jar in=data/HTTP/F775W out=... creator_did=... INDEX

java -jar tools/Hipsgen.jar in=data/HTTP/F775W out=... creator_did=... TILES

A color HiPS is generated from three distinct single-band HiPS, each one being assigned a color channel, red, green and blue.
Hipsgen syntax to create a color HiPS is:

java -jar tools/Hipsgen.jar inRed="..." cmRed="..." inGreen="..." cmGreen="..." inBlue="..." cmBlue="..."  RGB

Execute:

java -jar Hipsgen.jar inRed=./hips/F775W inGreen=./hips/F658N inBlue=./hips/F555W cmRed="-0.3 3     sqrt" cmGreen="0 8 sqrt" cmBlue="-0.4 5 sqrt" out=./hips/color creator_did=http/color RGB

Then, access the generated HiPS at http://localhost:2024/hips/color/.

Bonus exercise: adapt the cutout-params.csv file to generate color cutouts.