
	Weekly chat meeting #43: Thursday 15th Febuary 2001
	
Participants: Peter Kretschmar, Carol Anne Oxborrow, Niels Joergen Westergaard,
	Stefan Larsson
	
Subjects discussed: ISDC/MOC enumeration of JEM-X data taking modes
	New upgrade of DAL3JEMX - problems with version 3.4.0
	Pipeline script figures for ADD
	Parallelisation of SW processing in OGs
	Improved j_imag_basic_recon
	
Action Items:

AI43.1	PK	15/03/01 Circulate email about ISDC defined constants for
OPEN			JEM-X data-taking modes. SDAST to implement these
			constants in ISSW during V2 consolidation.


**** Logging Started : Thu Feb 15 09:30:23 MET 2001
<peter> I'm almost finished with the only one where I'm PI. Still waiting for comments from the CoI's though. END
> Hello Chaps! Nice to see so many of you here already. END
<peter> Good morning Carol Anne!
<peter> Are you also tied into any proposals or can you relax with writing ISDC software? END
> I'll try phoning Sami, if he isn't in we'll start without him. END
<larsson> Today I will only chat untill 10.00 -2 min since I have to
<larsson> be at another meeting at 10.00 (non-integral). END
> Okay, sami' not answering so let's get started. First what's this about
> data-taking formats being mapped differently Peter? END
> Hello? Anyone there? END
<peter> Long ago, ISOC came up with an enumeration of instrument modes
<peter> which they tried to have similar across instruments.
<peter> This enumeration is used mostly in the data sets exchanged between ISOC and us and also within OSM.
<peter> Normal data taking is usually 40-ish. So the JEM-X "Data taking" mode would be 40, but we added a second digit to encode the telemetry format used.
<peter> In the end-to-end tests, Mathias has now noted that one can in 3 out of 5 cases calculate this mode number by simpli adding 40 to a number found in the telemetry while for the other two one needs to switch: 2->43 and 3->42 (I ...
<peter> think).
<peter> Now Mathias has proposed to change this and all the fuss is to find out if any program from SDAST ever used these codes.
> I don't THINK we use these codes, but if we did it would probably be via a
> set of defined constants, meaning that you ISDC boys can play around with 
> the actual values as much as you like without it affecting us - assuming
> we've been good CTS-compliant coders (like we always are!)
> Does anyone else use these data-mode codes? END
<njw> I'll have to check this, because at some point I did use 41 etc for FULL, REST ...
<peter> What exactly gets stored in the MODE column of JMXi-MODE-STA? 
<njw> but I think that in the delivered ISSW it is not so END
> I'm just checking that Peter. END
<peter> The new enumeration is proposed as: 41=FULL, 42=SPTI, 43=REST, 44=TIME and 45=SPEC. Up to now 42 and 43 are switched. END
> I can't find right now what j_prp_status_table uses, but I think as general
> good practice  there should be defind constants in the ISDC library headers
> that code these values, and we can be totally oblivious of which values are
<peter> OK, let's just say that we have (or will have) such constants and these numbers will also be written to -INST-STA into the 
> actually used. I think Mathias' reason for making the new mapping is a very
<peter> MODE column. NJ: please remind me to get this into OSim as well. END
> good one since anything that streamlines/speeds up the very early processing
<njw> In dal3jemx.h it says: FULL=0, REST=1, SPTI=2 etc. We need new names for the proposed
> must be wellcomed. END
* njw constants. END
> Could ISDC put the new values into the next support-sw release, and we'll
> incorporate these constants into the V1/V2 software as part of general
> consolidation. I think consolidation should be for problems like this. END
> Perhaps the dal3jemx values should simply be re-worked. END
<peter> To NJ: sorry, When I wrote FULL=41, I did not mean constant names,
<peter> Before we get all confused, let me have a look and send out an email tomorrow, OK?
<peter> Before Stefan leaves, I have an URGENT announcement:
<peter> DO NOT USE DAL3JEMX 3.4.0!!!
> May we inquire why? END
<peter> Due to an unfortunate misunderstanding, Stphane implemented the wrong interface (integers instead of JMX1,JMX2)
<peter> in the new version of DAL3JEMX. We foundthis out when some V1 software suddenly behaved strangely.
<larsson> No problem I have'nt upgraded..... END
<peter> It will be corrected really soon, AFTER the proposal deadline.
> Thanks for warning us Peter - we could have done a great deal of
<peter> Probably noone is affected yet, but since the new Ref. Platform is just out, I thought, I better warn you now.
> `consolidation' before discovering this ourselves. END
<larsson> I have to leave now. Just like to say that Peters figures seems fine to me.
<larsson> END
<larsson> Bye. END
*** Signoff: larsson ()
> So we should wait for a new-new-reference platform release? END
<peter> Not necessarily a full RP release, but at least for DAL3JEMX 3.4.1. END
> Wait a moment, didn't Stephane leave ISDC? END
<peter> (or maybe 3.5.0). DND
<peter> He's still maintaining DAl3 from remote - you just encountered one of the problems of this ... END
> I see......END
> Okay, on to the figures: are we happy add-ers? END
> I will confess I'm leaving all ADD work until I've delivered q_identify_srcs
> which should be tomorrow, but the emails I've had seem promising. END
<njw> In that respect: yes. The figures that Peter has sent to me look fine - no
* njw problems END
<peter> Actually there is another point that came up at the Science Analysis Meeting here, ahich may affect our ADD somewhat:
> Sorry, q_identify_srcs get's delivered this morning, not tomorrow. END
<peter> - For performance reasons, ISDC would prefer to parallelize the analysis of the various ScWs in an OG.
> Meaning? END
<peter> This means that the analysis of one ScW is running on one processor, while that of the next runs already on another one.
<peter> - In principle this should pose no problems, BUT the way I have organized the "first level" scripts, like, e.g., j_image_bin
<peter> is somewhat orthogonal to this idea, as I have the ScW loop within these scripts always and an efficient parallelization
<peter> would rather have the ScW loop outside.
<peter> ISDC is still looking at the least complicated way to implement this scheme - ideally we would have it completely transparent to users
> So from an ADD sense, it's the script descriptions that need to be changed -
<peter> so they could profit from it if they have a suitable system but also run analysis strictly linear.
> does this include the fingures? END
<peter> Yes, exactly. I still need to get around to think about this and see what it means for me and what structure would be
<peter> best suited. 
<peter> END
> If you could put the current figures into CVS just as placeholders for me
> while you think about this I'd be really grateful END
<peter> No problem. They may be in there already - have you checked this? I just don't remember.
<peter> END
> Like I say all ADD's on hold till this afternoon or tomorrow. BTW while
> thinking about this datamode enumeration problem - if we have to
> wait for DAL3JEMX 3.5.0 (or whatever) couldn't the defined constants
> be implemented there with the new release and kill two birds with
> one stone? END
> Is that is now for figures and ADD - I'll get back to you on the
> j_correction figure ASAP, Peter. END
> Instead of trawling through the whole AI list, has anyone done anything
> that can be moved to AIlist.closed.txt? Anything at all????? END
<njw> I think not. Other things have had top priority. END
<peter> Well, since no-one requires redrwaing of those figures, AI 17.1 is probably closed at this point.
<peter> The new "parallel" redesign is open-ended. END
> Looking through the list I don't see anything else that strikes me as
> `done' - so on to AOB. AOB anyone? END
<njw> BTW, Peter, now that I'm typing, Niels Lund and I have found a way to get rid
<njw> of the ugly background structures in the JEMX reconstruction. This is done
<njw> by using a 'local' open fraction for the mask i.e. calculated for each skyimage
<peter> GREAT! CONGRATULATIONS!
<njw> point, rather that a global one. This will be implemented in the next
> Will this be implemented in j_image or as part of j_bkg_auto? END
<njw> delivery.
<njw> END
* njw It is in j_ima_basic_recon and therefore in j_image. END
> Great stuff NJW! END
<njw> Are we through now ? END
<peter> From my point of view, yes. END
> So chat with you next week - I think it's important to chat regularly
> now because it seems like a lot of bugs are being shaken out of the
> ISDC system at the moment. See you all next week! END
