Intro2: Working With MSs

1

Intro2: Working With MSs

2

Intro2: Working With MSs

3

Intro2: Working With MSs

4

Visibility Data & AIPS++ Measurement Sets

AIPS++... 2. AIPS++... [5] heard of it [2] tried to run it once [9] succeeded in running it once [5] have used it in anger [0] invented it 

On a Related Note... 2a. Reduction package of choice: [7] Classic AIPS [3] AIPS++ [2] Miriad [1] MeqTrees!!! [1] NEWSTAR [1] MabCal

AIPS++ is making great progress: at the previous workshop we had “tried”  “succeeded” –

(and one inventor that owned up to it all)

1

5

Intro2: Working With MSs

Working With Visibility Data 

MeqTrees interface with AIPS++ Measurement Sets



An “empty” MS has to be pre-fabricated using external tools:

–

– –

Meet Our Guinea Pig Skeleton 

other formats can be supported as necessary



you can use the AIPS++ “simulator”, see Workshop2007/demo_sim.g (and ask Tony) there's also a “makems” tool floating around (ask Ronald/Marcel/Joris)

This contains 27 antennas in VLA-C configuration...



So the max baseline is ~30km 8 hours observation, 5 minute sampling, 96 timeslots 32 frequency channels of 16MHz each, from 800MHz to 1.31GHz Four polarizations: XX XY YX YY One pointing

–





 

...but blown up by a factor of 10

8

Intro2: Working With MSs

VLA In Space (About demo.MS) 

I have prepared Workshop2007/demo.MS; this will serve most of our whims this week. There is a pristine backup copy available, so if you screw up, restore it with:

$ cd ~/Workshop2007 $ rm -fr demo.MS $ cp -a (/net/birch)/data/oms/Workshop2007/demo.MS . (/apps/Timba/data/oms/Workshop2007, if on jop01)

7

Intro2: Working 7 With MSs

6

Intro2: Working With MSs

A Simple MS Tree

    

Load Intro2/demo1-sink.py Under “TDL Exec”, select Tile size: 10 Load up the “MS Grids” bookmark Run “test forest” ...note the “history” slider in the visualizer

2

9

Intro2: Working With MSs

Why “Skeleton”? 







The VisDataMux

An MS provides a time/frequency grid (e.g., for use in simulations) – thus, “skeleton”: we ignore the data in the MS (and write our own) Sink nodes turn this grid into a request and send it up the tree. – one Sink per interferometer When a result comes back, this can be written out to a visibility column in the MS. MSs are processed in chunks of time called “tiles”.







11

Intro2: Working With MSs

A VisDataMux node was created for us automatically. The VDM is responsible for interfacing with the MS, reading data, and activating its child Sinks as appropriate. To start the process, we give a speciallyformed request to the VDM, containing input and output records telling it what and how to read (or write).

Sinks And Spigots 

 

 

Load Intro2/demo2-spigot.py Under “TDL Exec”, select Tile size: 10 Load up the “Spigots” bookmark, and the “Inspector” bookmark Run “test forest” ...note the “history” slider in the visualizer

12

Intro2: Working With MSs

We Can Read, Too!



10

Intro2: Working With MSs



A Spigot node reads the visibility data from an MS, and returns it as a visibility matrix Check visibilities using the history slider. –



You're looking at XX data, use the “Change selected Vells” option to look at the other correlations

You can probably guess what kind of observation demo.MS contains...

3

13

Intro2: Working With MSs

Inspector (Collections) Plot 







Matrices And Tensors

The last script introduced a “Collections” plotter (the ns.inspector node) A Meq.Composer node collects results from all its children into a single huge Result, which is plotted as a function of time. This plotter expects one data point per timeslot, so we use a Meq.Mean() node to take the mean in frequency. The inspector is attached as a special child to the VisDataMux node, labelled “post”. This makes it execute after (i.e., post) all the Sinks have fired. The result is published to the viewer (if active), then discarded.







X X XY Visibility data comes out as a YX YY 2x2 matrix In MeqTrees, this is represented by a Result with 4 VellSets, and a dims=[2,2] field: V0 V1 V2 V3 – no dims implies a 4-vector – and remember that each element can be its own function of frequency/time/etc. This can be generalized to tensors of arbitrary rank – e.g., the “inspector” node collects its children into a 351x2x2 tensor





15

Intro2: Working With MSs







An AIPS++ MS has three standard “columns” for visibility data: DATA, MODEL_DATA, CORRECTED_DATA. MeqTrees can “attach” to any column, or even create new columns. Tools like the AIPS++ imager assign specific meanings to these columns though, and do not support other names... Speaking of the imager, run this script:

$ glish -l make_image.g DATA ms=demo.MS





16

Intro2: Working With MSs

On MS Columns 

14

Intro2: Working With MSs

Let's Modify Some Data  

Load Intro3/demo3-mod-vis.py Here we apply a gain term: g p q =1.1 p e 2  iq /3



  



Under “TDL Exec”, select Tile size: 30 (the “go faster” option) Load up the “Inspector” bookmark Run “test forest” Switch inspectors to display complex phases (via right-click) Make an image:

$ glish -l make_image.g MODEL_DATA ms=demo.MS

4

17

Intro2: Working With MSs

Exercise 1: Freq-Dependent Gains

More Inspectors  

Here we have created two inspector nodes A Meq.ReqMux() node is used to feed a request to multiple children, we need it since we can only have one “post” child on a VisDataMux.

18

Intro2: Working With MSs

 

Start with Intro2/demo3-mod-vis.py Apply a frequency-dependent gain to the data: g pq=1.1 p

0  2 i 3q e 

0  

,

0 =8 108 ,  =5 108

tip: use Meq.Polar(x,y) to compose xe iy 

Make a per-channel image using:

$ glish -l make_image.g MODEL_DATA ms=demo.MS mode=channel

5

1 Visibility Data & AIPS++ Measurement Sets - GitHub

you screw up, restore it with: $ cd ~/Workshop2007 ... cp -a (/net/birch)/data/oms/Workshop2007/demo.MS . ... thus, “skeleton”: we ignore the data in the MS.

142KB Sizes 3 Downloads 78 Views

Recommend Documents

HW 2: Chapter 1. Data Exploration - GitHub
OI 1.8: Smoking habits of UK Residents: A survey was conducted to study the smoking habits ... create the scatterplot here. You can use ... Go to the Spurious Correlations website: http://tylervigen.com/discover and use the drop down menu to.

1 - GitHub
Mar 4, 2002 - is now an integral part of computer science curricula. ...... students have one major department in which they are working OIl their degree.

1 - GitHub
are constantly accelerated by an electric field in the direction of the cathode, the num- ...... als, a standard fit software written at the University of Illinois [Beechem et al., 1991], ...... Technical report, International Computer Science Instit

Simulated and Experimental Data Sets ... - Semantic Scholar
Jan 4, 2006 - For simplicity of presentation, we report only the results of apply- ing statistical .... identify the correct synergies with good fidelity for data sets.

Reasoning with Large Data Sets
Framework consisting of a collection of components which cover various aspects of ..... IRIS is an open source project developed under LGPL and available at:.

RECTANGULAR CONVEX SETS 1. Rectangular ...
Mathematics Department, California Institute of Technology, 1200 E. California Blvd, MC 253-37, Pasadena, CA. 91125. E-mail address: [email protected]

Data 8R Table Methods and Functions Summer 2017 1 ... - GitHub
Jul 18, 2017 - Data 8R. Table Methods and Functions. Summer 2017. Discussion 7: ... its range - the difference between the highest value in the array and.

Open Data Canvas - GitHub
Top need for accessing data online. What data is most needed? Solution. How would you solve this problem? ... How big is the universe of users? Format/Use.

Tabloid data set - GitHub
The Predictive Analytics team builds a model for the probability the customer responds given ... 3 Summary statistics .... Predictions are stored for later analysis.

Data 8R Plotting Functions Summer 2017 1 Midterm Review ... - GitHub
Data 8R. Plotting Functions. Summer 2017. Discussion 7: July 20, 2017. 1 Midterm Review. Question 4 ... function onto the table: Hint: Velocity = distance / time.

Data 8R Tables and more Visualizations Summer 2017 1 ... - GitHub
number of colds each volunteer gets. Is this an observational ... questions about it. A business has graphed the proportion of outputs in each year as a bar chart.

Data 8R Tables and more Visualizations Summer 2017 1 ... - GitHub
Jul 11, 2017 - At the same time, the researcher also records the number of ... A business has graphed the proportion of outputs in each year as a bar chart.

data tables - GitHub
fwrite - parallel file writer. SOURCE: http://blog.h2o.ai/2016/04/fast-csv-writing-for-r/ ... SOURCE: https://www.r-project.org/dsc/2016/slides/ParallelSort.pdf length.

Data Science - GitHub
Exploratory Data Analysis ... The Data Science Specialization covers the concepts and tools for ... a degree or official status at the Johns Hopkins University.