Project Executive Board meeting notes - extracts for WP3

14 Feb 2023

Carlo is collecting feedback for the KPI’s. An answer from DESY is still missing.

31 Jan 2023

No other activities than the final report are planned.

17 Jan 2023

WP3 is summarizing the KPI’s and preparing the session for the closing event.

20 Dec 2022

06 Dec 2022

Carlo is trying to hit the KPI targets. He will ask the other facilities about the ongoing use of the PaN search API.

22 Nov 2022

Carlo email:
“I think we can close this issue. Details are in the issue.

”

Majid asks when the deadline is. SOLEIL has problems with the endpoints. They have Scicat but they don’t have a real production-ready infrastructure. Patrick answers that this should be okay, because they can prove that it is technically done.

08 Nov 2022

Alun thanks the colleagues from SOLEIL for the hosting of the

WP3 Joint meeting. The meeting was very good and ensured the level of sustainability. 20 people were attending onsite and 25 people were attending online.

Kat will upload the recording on our website.

25 Oct 2022

WP3 is getting ready for the joint event with PaNOSC next week. There are approximately 20 people registered.

Carlo is working on #13, which is about onboarding our ontologies as an EOSC service.

The KPIs are monitored regularly. Alun asks DESY to work on the facility data catalogues to enable FAIR services and also on the PaN catalogue search endpoint.

11 Oct 2022

Paul, Antoine and Carlo discussed the PaN search API via email.

Only ten people have registered for the WP3 joint meeting between ExPaNDS and PaNOSC at SOLEIL. Majid will keep the registration open. Unfortunately Alejandra, Brian, Darren cannot attend. Also ALBA and ILL cannot attend.

Juliane asks who will care about the sustainability of the federated search API. Carlos answers that PaNOSC will do, but this topic will also be discussed at the joint event.

27 Sep 2022

Nothing to report.

13 Sep 2022

On the 2nd and 3rd November will be a joint PaNOSC/ExPaNDS WP3 meeting at SOLEIL

Carlo asks for a summary of the upcoming events. Juliane will attach them under AOB in the minutes.

30 Aug 2022

WP3 is working on a workshop with Majid, where SOLEIL should be the host.

Another workshop with PaNOSC is being planned.

16 Aug 2022

Carlo is collecting numbers for the KPI’s. Some facilities have gone backwards with their numbers in the survey, which must be validated. Carlo will do that and will catch the missing answers after his holiday in three weeks.

02 Aug 2022

Nothing to report.

19 Jul 2022

05 Jul 2022

Nothing to report.

21 Jun 2022

Nothing to report.

07 Jun 2022

Rolf suggested a workshop for the users of PIDs at the face to face meeting. Paul is needed for the discussion. There will be a hands on session (hackathon) from Max about the integration of the poral and the search API.

24 May 2022

WP3 meeting is on 25th May. Discussion will be about the topics of the f2f meeting. Carlos improved the search API portal.

10 May 2022

UmbrellaID workshop took place on 3rd May.

WP3 is helping other facilities with data catalogues and SciCat (e.g. DESY, SOLEIL).

26 Apr 2022

Nothing to report.

12 Apr 2022

Upcoming workshops: UmbrellaID (May 2022) Data catalogue

29 Mar 2022

Carlo needs input for the D3.4 from DAPHNE, Anton will ask Brigitte for the missing document.

Upcoming workshops:

Differences of SciCat and ICat (4th April 2022)

UmbrellaID (May 2022)

15 Mar 2022

Deliverable 3.3 was published on 16th March 2022.

Deliverable 3.4 will be sent out next week.

The onboarding of the ontologies as an EOSC service (#13) was discussed. It must be justified of ZENODO is an EOSC service. Kat will set up a meeting to discuss this issue.

01 Mar 2022

WP3 met with Ana to discuss training for WP3, which was helpful to prepare for the next upcoming deliverable. D3.3 on SciCat and ICAT demonstrators is being internally reviewed and finalised. It should be published by the end of the week.

About the issues, #13 on onboarding the PaNET ontology as an EOSC service needs to be addressed. About issue #21, Paul reminded the importance, for sustainability, of building a community around the ontology maintenance, starting by defining a clear release management (see related issue #46 in the ontology repository). Isabelle mentioned the NeXus community as a good role model to follow.

15 Feb 2022

PSI’s open data portal is now a registered service in the EOSC marketplace!

WP3 has been working on deliverable 3.3 which should be available tomorrow for internal review. It contains the description of the whole PaN search API stack deployed at MAX IV, examples implementations from ICAT and also the description of the ontology implementation, available independently of any catalogue.

Regarding the upcoming portfolio material for training, its content was discussed and it was agreed it should explain what material was already produced and what will come next, in particular the workshops. This should be discussed with Ana too, in line with the overall training plan of WP5.

01 Feb 2022

WP3 is focused on its upcoming deliverable to demonstrate ICAT and SciCat working with the PaN search API. MAX IV setup is to be used as a demonstrator. The deliverable will also cover the onboarding of a facility’s data catalogue in EOSC, linking to the associated milestone. Andy mentioned that STFC said in another meeting that the ICAT plugin for the PaN search API should be ready by the end of the week.

18 Jan 2022

07 Dec 2021

The catalogue gap assessment workshop preparation highlighted common areas of concern that WP3 can now focus on in the frame of tasks 3.4 and 3.5. The actual workshop happened this morning with all facilities represented.

Alun also mentioned the upcoming PaN search API scoring workshop to take place on Wed, 15th next week.

23 Nov 2021

In terms of resources for WP3, Brian mentioned Alejandra would be more involved again in the WP tasks. Nicolas added that a new person was hired at ALBA as MIS head, Beatriz. She will also be involved in WP3 tasks.

Sophie mentioned WP3 was working on the important upcoming data catalogue release due in February 2022, which will contain a module for the PaN search API including the implementation of PaNET in SciCat and Icat.

09 Nov 2021

Tasks 3.4 and 3.5 are now kicked-off and well taken over by MAX IV and HZB. Rolf sent to all facilities the request to review their status towards federated metadata catalogues, using the questions from the roadmap. A workshop will follow to discuss progress, issues and where WP3 can help.

Alun, Anton L., Patrick and Sophie also had a progress report meeting for WP3 last week, which Sophie will use to provide the achievements and next steps for each task.

Brian added Matthew arrived at STFC to replace Stuart who left WP3.

26 Oct 2021

12 Oct 2021

Anton L. is now the interim co-lead of WP3, in replacement of Sylvie. By January 2022, a more permanent representative will be appointed by Diamond. Isabelle added that an internal Diamond stakeholders meeting was scheduled in October to discuss resources.

The status of HZDR towards an OAI-PMH reachable endpoint for its metadata catalogue was updated.

28 Sep 2021

Following the last monthly management meeting with task leaders, tasks 3.4 and 3.5 are to be kicked-off tomorrow, including discussions on the interactions between those two.

HZDR discussed with B2FIND for the harvesting of their catalogue, which is in progress.

Note: the status of each facility is monitored here in GitHub - any updates welcome.

14 Sep 2021

Sylvie worked with Antoine to add material in the training catalogue for the PaN ontologies. EGI is still working with Bioportal for their onboarding process to the EOSC.

Recent talks with PaNOSC about the experimental techniques ontology (i.p. XFEL) will lead to the addition of some terms in the ontology. This will be the first “real-life” application of the maintenance process of the ontology and will help refine its governance (cf. issue #21).

Sophie asked about the progress on B2FIND harvesting and Uwe will inquire for HZDR. It may be done “easily” since their data catalogue (RODARE) is using CERN’s Zenodo which has a built-in OAI-PMH interface. Nicolas also mentioned his colleagues in WP3 at ALBA are working on it.

17 Aug 2021

Sylvie will leave Diamond and ExPaNDS in October 2021. Diamond is meeting early September to discuss her replacement.

Carlo has triggered task leaders to initiate the work on T3.3, T3.4 and T3.5. For the upcoming milestone, WP3 created a standalone and easy-to-install SciCat with associated documentation. It includes the search API with example data, including metadata following the techniques ontology. It will be useful for partners who don’t have a catalogue yet and for the training material to be developed in the frame of WP3. The document supporting the milestone is being finalised, adding input on what is planned also for ICAT.

20 Jul 2021

06 Jul 2021

Following the workshop, Alun had a meeting with all task leaders last week. They agreed on the schedule for the second period, on regular progress meetings and on dedicated kick-off meetings for the tasks that are starting now.

Carlo is working on the technical prerequisites to achieve the milestone due in August.

Sylvie reported the PaNET ontology v1.0 was now available in EGI’s OntoPortal (own instance of BioPortal set up by EGI for the occasion) which will now be onboarded to EOSC by Andrea.

22 Jun 2021

Alun reported that the half-day workshop he organised with Carlo for WP3 allowed for useful exchanges. Facilities like SOLEIL and MAX IV were able to make good progress already. Carlo is now collating the feedback and answers to the questionnaire evaluating the progress of each facility against the roadmap. Alun will follow-up with task leaders on the topics that were raised and plan for future meetings.

The ontology deliverable was published and presented by Steve at the EOSC symposium. The ontologies themselves are not yet available in EOSC but Sylvie and EGI are working on it (see issue).

25 May 2021

Alun and Carlo planned for the 18th of June a half-day workshop for WP3, to discuss the status and settle the work plan for the rest of the project.

The ontology deliverable (D3.2) was finalised and submitted for internal project review last Friday by Sylvie. The techniques ontology (PaNET) will be made public in parallel in the next few days. Brian will try and finalise his review next week, a few days delay is thus to be expected for the delivery to the EC. Sylvie reported Steve managed a submission for the EOSC symposium 2021 on this topic.

11 May 2021

D3.2 - Release V1.0 ExPaNDS ontology available as an EOSC online service (PSI) - 31 May 2021
This deliverable is being finalised.

MS12 - Metadata catalogue release (PSI) - 31 Aug 2021
Alun plans to set up a container with the data catalogue and the search API installed to demonstrate the use of the search feature on our data. This demonstrator should be useful in the overall roadmap.

27 Apr 2021

Sylvie reported recent work in WP3 was focused on writing the upcoming deliverable on ontologies. The ontology itself is called “PaNET” and will be made public in BioPortal right after the internal review.

A best practices meeting is being organised by ELI between 18-20th of May 2021 to share experience between partners on topics like user portals, data ingestion, logbooks and NeXus at PaN facilities. ExPaNDS has several contributions planned.

Sophie reminded Sylvie there was a milestone due for August 2021 called Metadata catalogue release that we should keep in mind.

13 Apr 2021

30 Mar 2021

In addition to the recent reporting activities, Sylvie mentioned the presentation organised last Friday on ontologies. Alejandra shared her general and long-acquired experience on the topic and Steve presented the update for the upcoming ExPaNDS deliverable. There was a good attendance and follow-up meetings between Brian, Alejandra and Steve are planned in the following days, to discuss e.g. relating our work to EMMO. PSI beamline scientists are also being involved via Carlo to provide feedback on what Diamond is preparing.

During last week’s NeXus codecamp, the script developed by ExPaNDS WP3 to extract NeXus classes and fields into an OWL was presented and well received by the NIAC. There will be a formal vote during the next assembly to accept it or not. If it is accepted, it will be hosted in NeXus Github and benefit from its sustainability and governance model.

16 Mar 2021

Alun confirmed PSI’s data catalogue would soon be harvested by B2FIND. They just need to ‘switch it on’, technically everything is ready.

Steve presented the NeXus ontology work last Friday to WP3. It can be found in GitHub, BioPortal (restricted, ask Sylvie for access) and WebProtege (accessible after sign-in). Its publication in open access will need to be approved by the NIAC. The techniques ontology is also available as a demo version on GitHub. A presentation to WP3 is planned on the 26th of March.

Sylvie reported that Andrea confirmed BioPortal was happy to be on-boarded to EOSC, which is very good news for us and also a good addition to the EOSC service portfolio. They are currently working on registering themselves as service providers. It will then be easy to link our PaN ontology as an EOSC “sub-service”.

FInally, the search API deliverable of PaNOSC is now finished, Sylvie contributed specifically to the ontology part.

02 Mar 2021

Alun reported progress implementing the ontologies developed at beamlines for some techniques. PaNOSC is also working on the demonstration of the search API where ExPaNDS is also contributing.

With regards to the ontology deliverable, Sylvie agreed to contact EGI to discuss further the feasibility of onboarding BioPortal as an EOSC service. It is indeed part of our upcoming deliverable that the ontology is made available in the EOSC portal, even if it is through another service.

16 Feb 2021

02 Feb 2021

19 Jan 2021

Sylvie asked for clarifications on what was precisely expected in ExPaNDS DoW (description of work) for the implementation deliverable called “Release V1.0 ExPaNDS ontology available as an EOSC online service”. A dedicated meeting will be held to clarify this with Patrick and Sophie.

Sylvie reported having successfully uploaded a draft ExPaNDS ontology in Bioportal. The PIDs to be used are still to be discussed. WP3 is working with NIAC to try and integrate in the NeXus terms the PaN ontology for the 1st classes. A demonstrator will be presented in the course of the next NIAC meetings.

Followed a discussion on the importance of having users check the usability of the ontologies in the long term. In particular, the structure of the ontology needs to easily accommodate changes for it to be sustainable.

15 Dec 2020

PaNOSC and ExPaNDS both have deliverables around the same time in the first quarter of 2021: a working federated search (PaNOSC) and a first PaN ontology (ExPaNDS). Alun reported they worked on making the two deliverables feed into each other and allow parallel work.

There is a sub-group in WP3 led by Steve Collins dedicated to ontology. He has a PhD student working on experimental techniques, who will be presenting his works to WP3 at the beginning of the new year. Sylvie added that they were working to turn the ontology, which is currently a NeXus spreadsheet, into a service: an actual repository with e.g. terms of use.

Alun also mentioned MAX IV reaching out to ask our community for implementation practices for DOIs. It was decided that the interested WP3 contributors would be invited to the first PID meeting of WP2 (on the 14th Jan), where the overall PID scene will be set. This will then allow for another meeting to follow-up more specifically on DOIs implementation.

01 Dec 2020

Alun reported WP3 now focused on the ontology deliverable due end of May 2021 and lead by Diamond. He also confirmed the assessment of the individual progress towards FAIR data catalogues presented in the status and roadmap deliverable (D3.1) was indeed to be used as a WP3 KPI. Sophie and Alun agreed to have the quotation presented on a 100 basis (instead of 10) and updated ~ every 6 months.

17 Nov 2020

27 Oct 2020

Patrick gave some feedback on the D3.1 deliverable on status and roadmap which Alun submitted to internal review last Monday. Sophie will send some more detailed feedback asap for a final version to be submitted to the EU at the end of the month.

13 Oct 2020

WP3’s upcoming deliverable is being finalised by the different WP3 contributors with the objective of submitting it to internal review by Monday next week.

29 Sep 2020

Alun and Sylvie received some input from partners for the landscape, to refresh the results from the questionnaire from Dec. 2019. The deliverable is progressing, including the resolved synergy with WP2 (see above).

Sylvie will upload the new version of the ontology to GitHub this Friday.

Sophie asked whether we could put the results of the WP3 survey to beamline scientists in the form of a ‘user story’ so we can better showcase it with our partner projects in the 5b and EOSC secretariat. Sophie is to send deadline and guidelines to Sylvie, as well as the link to the Interest Group on use cases.

15 Sep 2020

WP3 is progressing on the D3.1 deliverable, Sylvia reported that the December landscape was sent to facilities for update, with a deadline by Friday. The gap analysis and roadmap will then be finalised and sent for facility review before the end of the month.

In parallel, task 3.2 carries on with the shared meetings with PaNOSC on experimental techniques, the work-in-progress ontology will be uploaded to GitHub WP3.

The new starter at PSI will be taking up the implementation task (task 3.3) when he arrives mid Oct.

Tobias presented metadata at ExPaNDS and PaNOSC during the Workshop on Metadata catalogues integration for Interdisciplinary Research of FAIRsFAIR (11th of Sept) with the clusters (PaNOSC, EOSC-Life, ENVRIFAIR, SSHOC and ESCAPE). B2FIND was also presented and all material can be found here.

01 Sep 2020

Alun reminded the upcoming meetings on experimental techniques for T3.2. He also explained the outline of the D3.1 deliverable:

Alun is planning to distribute the latter template by the end of the week to be able to stick to an internal review of the deliverable starting end of Sept.

18 Aug 2020

Alun was invited to a ‘Metadata catalogues integration workshop’ by FAIRsFAIR and inquired WP2 about it. It will hold on the 11h of September.

The ontology work continues with two meetings planned by Sylvia in September with PaNOSC.

Alun is also working on the D3.1 deliverable which should be available for project review at the end of September.

04 Aug 2020

21 Jul 2020

Alun mentioned that D3.1 is depending on input from all facilities which was not so easy in this period. It is progressing still.

07 Jul 2020

Cecilia Blasetti from Elettra presented wayforlight to WP3 last week (01/07/2020). The survey results were analysed by Sylvie who will present them to WP3 tomorrow (08/07/2020). The raw data will be uploaded to ExPaNDS public GitHub repository if no one objects then.

23 Jun 2020

The user stories as raw results of the survey have been uploaded to the SharePoint by Silvia. An internal WP3 presentation of the analysis of the results is being organised. A presentation to a wider audience will be planned afterwards.

Works on the ontology are progressing, with the Pankos ontology being integrated into ExPaNDS’. The use of BioPortal to publish our ontology is currently being investigated, with the possibility to onboard the overall service into EOSC.

The Wayforlight presentation will be rescheduled for next week. Off-meeting: it is rescheduled for the 1st of July @10am CEST.

09 Jun 2020

The survey from T3.2 is now closed, it had 52 responses. The results will be anonymised and analysed for an internal WP3 presentation. The preliminary results are really interesting. In parallel, Silvia is investigating with other projects to see how they make their ontologies available. Alun mentioned he would be starting the M12 deliverable.

26 May 2020

WP3 has contributed to PaNOSC’s search API deliverable. Concerning the survey, Alun and Silvia mentioned 41 responses had been obtained, showing common trends. The deadline was extended one last time to the 5th of June to get more input, notably from MAX IV and ISIS (which is important to get a neutron source point of view).

28 Apr 2020

The WP3 workshop that was supposed to be held in M6 was not officially done like this but the objectives behind it were attained (clarification of the distribution of work between WP2, WP6 and WP3 on ontologies, coordination of activities with WP4). Patrick asked for a written justification.

Concerning the survey on search terms, the answers are still too “photon and Diamond centered”, but the partners are distributing it to their scientists (in progress at Soleil, Alba and HZDR). Sophie and Kat will also advertise it on EOSC platforms.

It raised the importance of explaining beforehand to our facilities scientists what ExPaNDS is concretely going to offer. In “COVID work mode”, it has been a bit complicated but Kat reminded what support was available in the SharePoint and Daniel and Alun are planning online presentations. Uwe mentioned he already used the poster to present ExPaNDS at HZDR.

14 Apr 2020

Recruitment update: 1.5 FTE to be filled by current job advertisements from PSI TBC for Diamond (Sophie to ask Silvia) 1 position to be filled by upcoming job advertisement from Soleil (update of a position open since January but that found no satisfactory candidate) Same as WP2 for ALBA

A meeting focused on management and reporting for WP3 is being organised this week by Alun, notably to feed the progress report. He also reported the search terms survey Diamond took with beamline scientists got 22 responses and was already being improved.

31 Mar 2020

Alun reported that the scientists survey on search terms was sent at the same time Diamond asked them to work from home. There are only few results and the deadline will be extended.

The work to clarify the general task on ontology is well advanced with regular meetings between WP2 and WP3. The minutes are available in the SharePoint (in Document Library/Meeting minutes).

17 Mar 2020

The current main task is on ontology. A survey to understand the search terms of the beam lines scientists at DLS will be sent in the following days, to get real users' input. They will have two weeks to answer and then after the results are analysed, it will be expanded to other facilities. The aim is to prioritize the ontology in the fields of most interest to scientists. A meeting with WP2 is also planned next week to work on the ontology.

03 Mar 2020

18 Feb 2020

The main achievements of the F2F meeting of WP3² in Lund are:

The distribution of T3.2 (ExPaNDS ontology) between the partners must be clarified with WP2 and WP6. Alun is seeing to it. It was reminded a similar work on ontology was made for the PaN data project.

04 Feb 2020

Reminder of the F2F joint meeting in Lund on 11-12 February.

21 Jan 2020

There will be a meeting next week to review the tasks and objectives of all WP3 participants. A F2F meeting with PaNOSC is also planned for February.

07 Jan 2020

03 Dec 2019

Recent works mainly on survey with WP2 and PaNOSC which was sent; mailing list updated; workshop in January with PaNOSC to be organized.

13 Nov 2019

Alun provided written input via e-mail for WP2 to the topic “WP reports” (13. November 2019 9:31):

Activity Date Comment
ExPands Kick off Sept 2019 Project start-up and
PaNOSC WP3 gathering Sept 2019
Expands WP3 mailing list setup End Oct 2019 Ongoing
PaNOSC Annual Meeting 4-7 Nov 2019 Including PaNOSC WP3 2 day breakout
Release of Joint WP2 & 3 questionnaire Mid Nov 2019
Request for task responsibilities sent Nov
Invited to HZB user meeting Dec 2019 link
Joint WP2 and WP3 Landscape analysis End Dec 2019
WP3 PaNOSC API and Data catalogue coordinating meeting February 2020 Copenhagen/Lund (Tobias Richter will take lead)