
	SDAST CHAT: Review of ADD 8.0 (draft version) 
	          Wednesday 19th November 2003
	          
Participants: Peter Kretschmar, Niels Jrgen Westergaard, Silvia Martinez-Nunez,
	Jerome Chenevez, Carol Anne Oxborrow, Stefan Larsson, Sami Maisala
	
Action Items: None


**** Logging Started : Wed Nov 19 12:58:12 MET 2003
<njw> I've noticed that many of your components have version 3.8.0 and delivery
> Hello Peter and NJW: perhaps I should give prizes for those who turn
> up first to the meeting? END
<njw> date 15/10/2003. Do you have a big chunk you ship off simultaneously ? END
*** silvia (~silvia@uhuru.dsri.dk) has joined channel #jemxadr
<njw> To Carol Anne: Indeed you should, at least today ! END
<peter> To NJW: Yes indeed - some smaller issues often get fixed in bunches with 
1-2 days of intense work all culminating in parallel.
<peter> To Carol Anne: I definitely agree - very good idea! END
> I wish I could say the same Peter (solving problems in bunches); all my
> components are so different that each one has its own special pathology.
> Hi Silvia! Hope you've getting everything ready for your visit next week -
*** Sami (~smaisala@gemini.astro.helsinki.fi) has joined channel #jemxadr
> we're very demanding guests you know. END
> Hi Sami! Jerome should be hear soon, he only stepped out to the kitchen. END
<silvia> Hi all ! Everything is ready for your visit except me - I caught the 
fluEND
> hear==here END
<Sami> Hello everybody.END
> Do they have Spanish flu in Spain, or is it called English `flu there? END
<peter> To Silvia: sorry to hear that - I hope you'll go home and treat yourself 
right after the chat. END
*** larsson (~larsson@tenma.dsri.dk) has joined channel #jemxadr
> Hi Stefan! We're all here except Jerome, who's just coming. Silvia has
<njw> Hello Silvia, Sami and Stefan END
> permission to rest a bit as we chat. END
*** jerome (~jerome@uhuru.dsri.dk) has joined channel #jemxadr
<silvia> Is a quite silly spanish flu, only attacks to the stomach , but in a 
very unpleasant way ! END
> We're all here so the party can begin.
<njw> Same kind a Paul has (had?) ? END
<silvia> I am afraid yes, NJW END
> Are we agreed with Peter's procedures and my suggestion to go chapter
> by chapter? END
<jerome> Yes. Hello everyone! END
<njw> Y
<njw> Hi Jrme END
> Hello Jerome! END
<peter> Shall we begin? END
> If you're all sitting comfortably, then I'll begin.
> First I'd like to say that I hope you've all done all the work mentioned
> in the blizzard of emails I sent you in the last couple of weeks - don't
> have time to go through every component, and I don't always know what
> inputs/output/parameters/functions they use.
<peter> Sure ... we would never dare not to do our homework :-) END
> That's not quite true Peter, I'm still clearing up rampant AUXL-ORBI-NRTs
> in the document.
> Second point: please give me your corrected paper copies of the document
> in spain so that I can merge the minor corrections when I get home.
> NJW and JC are excused hauling the ADD to spain first.
> So chapter one: any big points? END
<jerome> N
<peter> No
<silvia> No
<Sami> No
<njw> A rephrase of a sentence and some additions to the acronym list. I'll 
email. END
> Peter, this morning you mentioned that V1 is now known as Technical
> processing. Have V2 and V3 been given names too, and if so what? END
> No need to email, just give me your corrected copy NJW. END
<peter> From what I see here, V2+V3 is called Scientific Processing for anything 
connected to SA/QLA/OSA.
<peter> I think in many cases we could just drop the V2/V3 association without 
harm.
> Shall we continue with the terms V1, V2, V3 or are these obsolete? END
> Okay, will do. END
> So chapter 2: SRD. I've only one comment. SR 18.8.3 is the only one which
> isn't definitely discarded or definitely satisfied. It reads:
> J-IMAGE-GENERATION, Consistency with other instruments: If possible,
> the image data structure should be consistent with those used by IBIS and
> SPI.
> How do we stand here? Is consitent the same as identical, or just similar?
> END
<peter> Similar is enough. And that is definitely fulfilled. END
> As for 18.20.3 (mosaicing), will we have a JEM-X specific mosaicing tool
> or is it assumed that we'll always use the ISDC generic tool? END
<peter> My preference is the latter - simply to save a lot of effort. As you 
have seen, Roland has made a lot of progress.
<peter> This would probably work with the normal ISSW images, if we had variance 
images in there as well. END
> So for the time being we assume that our OBS tools (basically mosaicing)
> are taken care of by ISDC? END
*** Signoff: larsson ()
*** larsson (~larsson@tenma.dsri.dk) has joined channel #jemxadr
> Any more comments on SRD chapter? END
<larsson> (Just switched computer for a more relaxed working position) END
<njw> N
<Sami> Should we add j_bkg_scal_calc also in 18.7.x ? END
<Sami> sorry 18.17.x END
<peter> Probably yes - you figure out where it matches best. END
> Yes Sami, I think you're right, I've listed it for 18.7.1, 18.7.2 and
> 18.7.3 in the component description. END
> Any more comments on requirements? END
<Sami> N
<silvia> N
<larsson> n
<jerome> N
<njw> N
> On to chapter 3: comments please END
<njw> I have two changes in 3.1 that need general agreement (I think):
<njw> p. 15, 4th bullet: -> To make shadowgrams with corrected event position
<njw> and energy.
<njw> and energy.
<njw> Same page: Remove last bullet (not a job for science analysis software).
<njw> Section 3.6 "The background problem"  2nd paragraph, needs a rewrite END
<njw> PS sorry for repeating END
<peter> I agree with the changes. END
<njw> They'll be handed over to Carol Anne, then. END
> For that last bullet, how about "Monitor corrections to see whether
> tables need updating" I'm thinking of j_ical here. END
<njw> Yes, that's fine END
<peter> OK
<jerome> What about dead anodes monitoring? END
> Sami, can you do the rewrite on the background problem, I think that's
> your contribution. END
<njw> Isn't that included in "perform a running check on the instr.'s 
performance" ?
<jerome> OK, I see. END
<Sami> Yes. I can do that. Please send me a comments of the messy parts. END
<njw> I will END
<Sami> Figure 3.3 (p. 20) has still old bkg data structures. END
> Peter, figures 3.1, 3.2 need to show that JMXi-GAIN-CAL-IDX goes into
> j_correction and that HK goes into j_dead_time. Have you still got the
> pages I faxed to you a couple of weeks ago? Also figure 3.3 doesn't
> seem to be updated at all. END
<peter> Changes to figure 3.1/3.2 - OK.
<peter> For figure 3.3 I do not have the .fig file - as I told you in a mail.
<peter> Here the question is also if this one shouldn't be broken up into 
different figures:
<peter> (1) A figure showing the full functionality of OSA (Fig 3.1+additions) 
and one showing all the other interactive parts. 
<peter> Hmm, there should have been a (2) somewhere ...
<peter> For the latter, I would need some clear descriprion of your tools - can 
I assume that the current OSA text is OK? END'
> Yes I was going to suggest a figure that shows the pipelines and the
> observational stuff. Also, there was a suggestion some time ago to
> show how the ISDC levels branch after a certain point. If you could make
> a new diagram to replace 3.3 that contained all these elements it would
> be good. END
> Or even two figures END
> Next: I think you need to write somewhat more fully on background
> handling in the imaging, binning and source extraction stages, Peter, NJW and 
Stefan
> We really need to know how you employ sami's output and whether there
> are occassions when you don't use it at all. END
<silvia> I completely agree with Carol Anne !END
<larsson> Accepted. END
<njw> Alright, I'll write something for j_ima_shadowgram END
<peter> I'm not sure, I thought I said everything I had to say - can you explain 
what is missing in my case?
<peter> END
> If there aren't any more details that's fine - things haven't become
> more complicated in the last year? END
> My next question is whether we should include jemx.h as an appendix to
<peter> No - as I 'just' bin Sami's data. END
> the ADD or not. `Just'
> binning is fine if that's all you do. END
<peter> W.r.t jemx.h - it would make sense as an appendix.
<silvia> To PK: sorry Peter ! you are right, your section about bkg handling is 
clear END
> Okay, next point: I'm updating figures 3.5, 3.6 and 3.7 with all PK's
> suggestions and any others I receive between now and next thursday.
> Also j_data_preparation is being purged. I've changed your j-data-preparation
> description Peter, to make it a `package' and include the Rev. file pipeline
> components too. j_sci-format is being purged throughout too, and in this
> regard I think Stefan needs to write a piece for the detailed design
> chapter describing the different data structures he produces (PHA1 PHA2)
> and explain how these are needed for generic tools. END
> Any comments? END
> Finally, have we changed our ideas about dream tools? Have we become
> more realistic, or are we dreaming of even more wondrous tools in
> the future? END
<peter> Well, the current description is very generic. I'd drop the last bullet 
which doesn't describe anything.
<peter> Should we mention a true multi-pointing analysis here?
<peter> What about a j_ima_src_find++ working on mosaics? END
> On the contrary it describes far too much! I'll drop it - we don't want
> to invite PIs to give us lots of work. Yes j_ima_src_find for mosaics
> sounds good, but won't this be covered by some generic ISDC tool? END
<jerome> What about the ISDC special tools like pick_spe, pick_lc...? END
<njw> I'm also more inclined to put here only a very limited list of components
<peter> To CA: There is no generic ISDC tool to search for sources as the 
knowledge of what peak could be releveant seems rather specific.
<peter> Maybe we'll have one someday but not yet. END
* njw more or less ready. Better to document facts than dreams. END
<peter> To Jerome: spe_/lc_pick and there's nothing special about them - they're 
described in our general documentation.
<peter> Do you want to include them in the ADD? Could be done for the OSA part. 
END
<njw> On source finding: It's true - the PSF depends a lot on the input images 
END
> No I'm not looking to include more of the ISDC components now - only
> those that are deeply inbedded in our pipeline. I'll add the source
> finding tool as a dream item. END
<jerome> I mean, maybe they could be mentioned there if they fulfill at least a 
part of 
<jerome> this multi-pointing analysis we are talking about. END
<silvia> I am sorry but my stomach is "shouting" me. I am going home. I hope all 
of you will have a nice trip to Spain. See you on Monday. END
> If there are no more comments on chapter 3, let's go on with chapter 4
*** silvia has left channel #jemxadr : (silvia)
> Detail Design.
> Take care Silvia, see you on Monday. END
> First I'd like Stefan to write a bit on PHA1/PHA2 output data formats and
> extend his `Usage with XSPEC' to `Usage with XSPEC and XRONOS' - this
> is really to cover the bald patch where j_sci_format once was END
> All comments about chapter 4 welcome. END
> I'd also like to include two figures showing the current ADC/PHA and
> PI/KeV mappings if no one minds. It's something I've often found useful END
<larsson> I can add something more there CA although some of the j_sci_format
<larsson> is now in the isdc tools. I will try to explain a bit more regarding
<larsson> the use of XSPEC and XRONOS with our data products. END
<peter> A good idea. END
> That would be great, thank you, Stefan END
<njw> Figure 4.1 is now obsolete, I think. Arrays of that type assume raw
<njw> event positions. The 7.05/7 mm pixel size has been given up. END
<njw> The section 4.7 needs a rewrite when the new convention has been 
implemented. END
> I'll scrap the figure, but can you then pad out your section on
> detector positions? END
<njw> OK END
> Don't you mean section 4.5 is obsolete, that's the detector positions
<larsson> Stop!
> one. If section 4.7 is obsolete JC needs to know and needs to do a
> rewrite.  END
<larsson> I am not sure I agree about that figure.
<larsson> I do use that in my software so maybe it is good to have it
<njw> Sorry Stefan, perhaps we need to discuss this in Valencia END
<larsson> described in the ADD even if there is no direct refernce.
> But if the 7 pixels to every 7.05mm of collimator grid is obsolete, so
> is your software. END
<jerome> I was ready to ask: Should the part 4.7 Flux Units not be modified 
regarding the recent
<jerome> discussion we have had about normalisation?
<jerome> Now, what is the status?
<jerome> END
> I think it should. END
<larsson> OK that part I dont use. so in that sence I agree.
<njw> To Jrme: yes END
<peter> Yes indeed
> I'll remove the figure. I also have responsiblity for updating the
> detector posistions section, 4.5 (I'll talk to you aobut this NJW) and
> JC updates 4.7. NJW do you need to update the piece on spectral data
> handling? END
<njw> What piece ? END
* njw You mean section 4.6 ? END
<larsson> I will go through 4.6 to make the update with regards to
> Yes, 4.6, that's yours for some dark historical reason. END
<larsson> xspec and xronos use etc. END
<njw> Thanks, Stefan END
<njw> Section 4.6.2 seems to be in order END
> Section 4.6 is in NJW's directory at add/njw/SpectralData.tex END
> That's all I've got on chapter 4, now on to component descriptions,
> any one want to get the ball rolling with general remarks? END
<njw> Stefan, I'll mail a copy to you END
<larsson> (Fine NJ) END
> Alright, I'll start:
<njw> Should we agree on a common way to give dates ? END
<peter> We could.
> Many many components lacking real resource estimates: code size, CPU time, 
memory required for the data segment etc.etc. We were asked to do this almost 2 
years ago.
<njw> Is it still important ? END
> Resources: None    or Resources: No unusual    is not very helpful. END
> It was important then, why isn't it important now? END
<njw> Machines grow bigger END
<larsson> That is the question. END
<larsson> Also, the software is running. The idea was probably to see if
<larsson> any problems were expected.? END
<larsson> And Peter says? END
<peter> Yes, indeed. If we had, e.g., decided that we do background handling
<peter> with the 'gigacube' approach, that would have been important 
information.
<peter> "
<peter> Sorry, just messed up my input.
> But we could apply Stefan's logic to the entire ADD: if ISDC's got the
<peter> Should we replace "No unusual" by another wording indicating only 
insignificant resource usage?
> software running now, why the documentation, why not just look at the
> code, the results, the unit tests etc. END
<peter> The ADD is supposed to help understanding the code.
<njw> I don't consider that piece of information important to understand the 
code END
<peter> It should be a useful document for any newcomer.
> Is the consensus to scrap the resources field, or simply fill it
> with something generic instead of giving real numbers? ENd
<njw> The latter END
<peter> I'd leave the existing real data, where we have it. Otherwise the 
latter.
<njw> Then there is an option to flag some unusually big memory demands END
<larsson> The latter unless there are significant requirements. END
<peter> So we all agree - where does "unusually big" begin? 256 MB, 1 GB? END
<njw> With our current ISSW I think we have nothing "unusually big" END
> Okay, I think `No unusual requirements' will fit most cases.  Presumably
> it's the big instruments that have big demands. END
<njw> Should we insist on a person name when just an institution is given as
<njw> responsible for a component ? END
> Yes I think a person should be responsible for each, and Mr Newcomer should
> know who to contact with questions. ENd
<njw> I agree END
<peter> ok
<larsson> ok
> Everyone should put their names on their comonents. ENd
> My next general point is that there are several components with no
> design remarks. After several years of evolution for each component this
> strikes me as a little strange. Please add remarks to all your components:
> algorithms, structure, input anomalies, output quirks etc. etc. Why
> you did it the way you did. Don't forget to mention all the times ISDC
> forced and inelegant solution on us! (Just kidding, sort of, Peter) END
<peter> I have not even ignored that :-) END
> Please remember this is also the DDD so we need D (details) END
<peter> Anything more here?
> Any comments? ENd
> Shall we move on to the Data Dictionary? ENd
<peter> The main point is that really some details should be added instead of 
remaining unaccessible in the more or less porous grey matter between the ears 
,... END
<peter> Yes, let's move on. END
<jerome> Good idea!
> I have no particular remarks about the data dictionary, what about the
> rest of you? END
<peter> Neither do I. It is actually a useful resource. END
<jerome> N
<larsson> No
<njw> N
<Sami> N
> Okay, the bibliography has been updated and a couple of documents added
> since last week. Any comments? ENd
<jerome> I suggested to add ISDC JEMX Analysis User Manual. Is it OK? END
<njw> No, except that you've done a good job, Carol Anne, considering that
> Yes, that's been overlooked, it's a good suggestion END
<njw> we've only spent 2 hours and we're approaching the end END
<peter> no comment
> Finally, the index. I've added a mass of new indexing commands throughout
> the document so that the index is a complete ISSW description in itself.
> Mr (or Ms) Newcomer should be able to look up in the index any data
> structure, input parameter, component and see how it's used by all the
<njw> That index is really helpful. END
> other components/parameters/data structures. There were some bugs in the
> indexing, but with JC's sharp eyes I've ironed them out and the final
> version should be even more comprehensive (and even bigger!) END
<jerome> Should we have an index for the index...?
<jerome> Just kidding :-) END
> I hope you are just kidding! Any more (more constructive) comments? END
<peter> No
<njw> Would it be worthwhile to put a small paragraph with advice for this 
famous
<larsson> n
<njw> newcomer on how to read this document such as the comprehensive index and
<njw> having the JEMX user manual in the other hand? END
> I can add such a comment at the beginning of the `Scope of the document'
> section in the first chapter. END
<peter> Good idea - something like a Quick Guide to this Document.
> Well that's it for the document. Can I remind you all (except NJW and JC)
> to bring your corrected copies to Valencia. I won't be doing any more on
<njw> Peter, I have a list of comments for you. Is it sufficient to bring it
> the ADD this week because I'll be preparing for the meeting, so all
<njw> to Valencia ? (Easier than writing an email) END
> updates can safely be left after the meeting. END
<peter> To NJW: yes that should do. END
> One last thing: is the Valencia agenda to everyone's liking? END
<jerome> Yes. I will just go to bed before 3 o'clock I guess. END
<njw> Agenda looks OK END
<larsson> Y
<peter> The agenda is OK
<Sami> y
> So there's only one more thing to say: See you all in Sunny Spain! END
<njw> Have a nice trip. Bye END
*** njw has left channel #jemxadr : (njw)
<Sami> See you! END
> Bye
<jerome> See you there. END
> !
*** Signoff: Sami ([x]chat)
*** jerome has left channel #jemxadr : (BYE)
