From d3171a047de243f036cc508ff48588bb0e029eff Mon Sep 17 00:00:00 2001 From: Heidi Schellman <33669005+hschellman@users.noreply.github.com> Date: Sat, 26 Jul 2025 09:37:48 -0700 Subject: [PATCH 1/5] link to computer setup --- setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.md b/setup.md index ebd09dd..7ffa408 100644 --- a/setup.md +++ b/setup.md @@ -83,7 +83,7 @@ Also check out our [Computing FAQ](https://github.com/orgs/DUNE/projects/19/view ## 0. Basic setup on your computer. -[Computer Setup]({{ site.baseurl }}/ComputerSetup.html) goes through how to find a terminal and set up xwindows on MacOS and Windows. You can skip this if already familiar with doing that. +[Computer Setup]({{ site.baseurl }}/ComputerSetup) goes through how to find a terminal and set up xwindows on MacOS and Windows. You can skip this if already familiar with doing that. > ## Note > The instructions directly below are for FNAL accounts. If you do not have a valid FNAL account but a CERN one, go at the bottom of this page to the [Setup on CERN machines](#setup_CERN) section. From e413180f3844cac7606da59cf4f89c63e1215cd3 Mon Sep 17 00:00:00 2001 From: Heidi Schellman <33669005+hschellman@users.noreply.github.com> Date: Sat, 26 Jul 2025 13:04:43 -0700 Subject: [PATCH 2/5] clean up broken links --- _episodes/02-storage-spaces.md | 2 +- _episodes/03.2-UPS.md | 2 +- _episodes/05-end-of-basics.md | 11 +++-------- _extras/ComputerSetup.md | 2 +- setup.md | 2 +- 5 files changed, 7 insertions(+), 12 deletions(-) diff --git a/_episodes/02-storage-spaces.md b/_episodes/02-storage-spaces.md index 444f452..38aab00 100644 --- a/_episodes/02-storage-spaces.md +++ b/_episodes/02-storage-spaces.md @@ -295,7 +295,7 @@ root -l root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio ~~~ {: .language-bash} -See the next section on [data management](({{ site.baseurl }}/03-data-management)) for instructions on finding files worldwide. +See the next episode on [data management]({{ site.baseurl }}/03-data-management) for instructions on finding files worldwide. > ## Note Files in /tape_backed/ may not be immediately accessible, those in /persistent/ and /scratch/ are. {: .callout} diff --git a/_episodes/03.2-UPS.md b/_episodes/03.2-UPS.md index 9b031cc..b463a1e 100644 --- a/_episodes/03.2-UPS.md +++ b/_episodes/03.2-UPS.md @@ -17,7 +17,7 @@ keypoints: > You need to be in the Apptainer to use it. > UPS is being replaced by a new [spack][Spack Documentation] system for Alma9. We will be adding a Spack tutorial soon but for now, you need to use SL7/UPS to use the full DUNE code stack. > -> Go back and look at the [SL7/Apptainer]({{ site.baseurl }}setup.html#SL7_setup) instructions to get an SL7 container for this section. +> Go back and look at the [SL7/Apptainer]({{ site.baseurl }}/setup.html#SL7_setup) instructions to get an SL7 container for this section. {: .challenge} An important requirement for making valid physics results is computational reproducibility. You need to be able to repeat the same calculations on the data and MC and get the same answers every time. You may be asked to produce a slightly different version of a plot for example, and the data that goes into it has to be the same every time you run the program. diff --git a/_episodes/05-end-of-basics.md b/_episodes/05-end-of-basics.md index 20f3b64..23f8478 100644 --- a/_episodes/05-end-of-basics.md +++ b/_episodes/05-end-of-basics.md @@ -31,11 +31,6 @@ Learn about `bash`, `github`, `python`, `cmake`, `root` and many more HEP comput - [The LArSoft tutorial at CERN, February 3-7, 2025](https://indico.cern.ch/event/1461779/) password on the [tutorials page](https://wiki.dunescience.org/wiki/Computing_tutorials) -- [Make your code more efficient]({{ site.baseurl }}/05.1-improve-code-efficiency.md) -- [The MRB build system]({{ site.baseurl }}/05.5-mrb.md) -- [How to modify and rebuild LArSoft Modules]({{ site.baseurl }}/06-larsoft-modify-module.md) -- [Grid job submission]({{ site.baseurl }}/07-grid-job-submission.md) -- [Grid job submission with justin]({{ site.baseurl }}/08-submit-jobs-w-justin.md) -- [Debugging grid jobs]({{ site.baseurl }}/09-grid-batch-debug.md) - ---- \ No newline at end of file +- [Make your code more efficient]({{ site.baseurl }}/05.1-improve-code-efficiency) +- [LArSoft basics](https://dune.github.io/computing-basics-larsoft) +- [Batch submission basics](https://dune.github.io/computing-basics-batch) (coming soon) diff --git a/_extras/ComputerSetup.md b/_extras/ComputerSetup.md index 7875178..a8c86dc 100644 --- a/_extras/ComputerSetup.md +++ b/_extras/ComputerSetup.md @@ -83,7 +83,7 @@ See the information about [Windows]({{ site.baseurl }}/Windows.html) terminal co - Alternatively, if you have access to a remote linux system through your institution you can use the Windows terminal/X-windows connections described in [Windows]({{ site.baseurl }}/Windows.html) to connect to that system and work there. > # Note -> You should now be ready to go for the ({{ site.baseurl }}/setup.html) +> You should now be ready to go for the ({{ site.baseurl }}/setup) {: .callout} ## Extra - Get a compiler/code editor diff --git a/setup.md b/setup.md index 7ffa408..59a0281 100644 --- a/setup.md +++ b/setup.md @@ -670,7 +670,7 @@ fire up the Apptainer as explained in [SL7 Setup](#SL7_setup) but with a slightl You may have to add some mounts - here I added `/afs/` but removed `/nashome/`, `/exp/`, `/etc/krb5.conf` and `/pnfs/`. -You should then be able to proceed with much of the tutorial thanks to the wonder that is [`/cvmfs/`]({{ site.baseurl }}/03.3-cvmfs.html). +You should then be able to proceed with much of the tutorial thanks to the wonder that is [`/cvmfs/`]({{ site.baseurl }}/03.3-cvmfs). Set up the DUNE software From d0385e4b09eb6eed7f056ae31f2e89d9d2ad5041 Mon Sep 17 00:00:00 2001 From: Heidi Schellman <33669005+hschellman@users.noreply.github.com> Date: Sat, 26 Jul 2025 20:01:54 -0700 Subject: [PATCH 3/5] add in note about rucio config warning message --- _extras/al9_setup_2025.md | 13 +++++++++++-- setup.md | 14 +++++++++++++- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/_extras/al9_setup_2025.md b/_extras/al9_setup_2025.md index 7cc8ccc..05e8d3a 100644 --- a/_extras/al9_setup_2025.md +++ b/_extras/al9_setup_2025.md @@ -21,6 +21,15 @@ export RUCIO_ACCOUNT=${USER} ~~~ {: .language-bash} -You can ignore the warning messages - this is still under development - +You can ignore most warning messages - this is still under development - but pay attention to this one. +> ## Note +> > ## You may see a rucio config error message that looks like this +> > ~~~ +> > ==> Warning: Your rucio config /nashome/X/XXXXX/.config/rucio/dune/etc/rucio.cfg does not list oidc authentication +> > If you remove it and spack load r-m-dd-config again it will get regenerated with oidc (Token) authentication +> > ~~~ +> > {: .output} +> > This happens if you've run older versions of rucio that used kx509 authentication. Remove `$HOME/.config/rucio/dune/etc/rucio.cfg` as it advises and rerun the `spack load r-m-dd-config` command to reset the config file. You should not need to do this again. +> {: .solution} +{: .callout} diff --git a/setup.md b/setup.md index 59a0281..5831b03 100644 --- a/setup.md +++ b/setup.md @@ -352,6 +352,18 @@ spack load justin ~~~ {: .language-bash} + +> > ## You may see a rucio config error message that looks like this +> > ~~~ +> > ==> Warning: Your rucio config /nashome/X/XXXXX/.config/rucio/dune/etc/rucio.cfg does not list oidc authentication +> > If you remove it and spack load r-m-dd-config again it will get regenerated with oidc (Token) authentication +> > ~~~ +> > {: .output} +> > This happens if you've run older versions of rucio that used kx509 authentication. Remove `$HOME/.config/rucio/dune/etc/rucio.cfg` as it advises and rerun the `spack load r-m-dd-config` command to reset the config file. You should not need to do this again. + +{: .callout} + + > ## Optional > > ## See if ROOT works > > Try testing ROOT to make certain things are working @@ -662,7 +674,7 @@ log into `lxplus.cern.ch` fire up the Apptainer as explained in [SL7 Setup](#SL7_setup) but with a slightly different version as mounts are different. ~~~ -/cvmfs/oasis.opensciencegrid.org/mis/apptainer/current/bin/apptainer shell --shell=/bin/bash\ +/cvmfs/oasis.opensciencegrid.org/mis/apptainer/current/bin/apptainer shell --shell=/bin/bash \ -B /cvmfs,/afs,/opt,/run/user,/etc/hostname --ipc --pid \ /cvmfs/singularity.opensciencegrid.org/fermilab/fnal-dev-sl7:latest ~~~ From 8ced0e56a4b54bd68b7e2d7f914403114224e28a Mon Sep 17 00:00:00 2001 From: Heidi Schellman <33669005+hschellman@users.noreply.github.com> Date: Sun, 27 Jul 2025 13:50:34 -0700 Subject: [PATCH 4/5] add in documentation episode --- _episodes/01.5-documentation.md | 80 +++++++++++++++++++++++++++++++++ index.md | 12 +++-- 2 files changed, 88 insertions(+), 4 deletions(-) create mode 100644 _episodes/01.5-documentation.md diff --git a/_episodes/01.5-documentation.md b/_episodes/01.5-documentation.md new file mode 100644 index 0000000..1389822 --- /dev/null +++ b/_episodes/01.5-documentation.md @@ -0,0 +1,80 @@ +--- +title: DUNE Documentation +teaching: 5 +exercises: 0 +questions: +- Where can I find documentation for DUNE +objectives: +- Learn how to get access to DUNE documentation +- Learn how to find DUNE documentation +keypoints: +- There is documentation somewhere! +--- + +## Documentation access + +Much of DUNE's computing documentation is public and hosted in github + +But some requires access via Fermilab SSO or CERN credentials. Documentation is protected for several reasons. + +1. it includes specifics about computing systems that might assist hackers + +2. it contains preliminary information about scientific or technical projects that have not been cleared for publication + +3. it contains confidential details about contracts/purchases + +4. it just ended up there + +We strongly suggest that DUNE collaborators obtain both [Fermilab computing accounts](https://get-connected.fnal.gov/users/access/) and, at least, a [CERN Guest account](https://cern.service-now.com/service-portal?id=kb_article&n=KB0007539). + +Those actively engaged in running prototypes at CERN likely need to get full [CERN Neutrino Platform computing access](https://neutrino-secretariat.web.cern.ch) and join the appropriate egroups. + +## DUNE tools list + +[https://atwork.dunescience.org/tools/](https://atwork.dunescience.org/tools/) +lists a large number of DUNE data repositories. + +This document highlights a sub-sample and how to access them + + +## Docdb (Requires FNAL SSO) + +Docdb is the main repository for analysis notes and physics results. + +[https://docs.dunescience.org](https://docs.dunescience.org) + +New DUNE collaborators should be given access automatically and can then use the +[SSO Link](https://docs.dunescience.org/cgi-bin/sso/DocumentDatabase) + +if that isn't working, use the [Apply for access](https://docs.dunescience.org/cgi-bin/sso/CertificateApplyForm) link on the main page. + +## DUNE wiki (Requires FNAL SSO) + +The DUNE wiki is at [https://wiki.dunescience.org](https://wiki.dunescience.org/) + +New users should be automatically added to the access list. If you can't access the wiki please send mail to [dune-communications@fnal.gov](mailto:dune-communications@fnal.gov) specifying the error you see. We've found that changing your email address can result in access problems. + + +### Tutorials list on the wiki + +[https://wiki.dunescience.org/wiki/Computing_tutorials](https://wiki.dunescience.org/wiki/Computing_tutorials) is a list of available tutorials. + +## CERN EDMS + +Many technical specifications are stored in the CERN EDMS system + +[https://edms.cern.ch/](https://edms.cern.ch/) + +Most files are accessible without login but submitting documents requires a CERN computing account. + +## Github repositories + +DUNE has a Github organization: [https://github.com/DUNE/](https://github.com/DUNE/). Repositories in that organization are generally public to read but you need to request access through [dune-communications@fnal.gov](mailto:dune-communications@fnal.gov). + +Many repositories have wikis or associated dune.github.io pages. + +## DUNE Computing FAQ + +[Computing FAQ](https://github.com/orgs/DUNE/projects/19) + +Lists of common connection problems and issues with running jobs. \ No newline at end of file diff --git a/index.md b/index.md index 78896f7..56869f0 100644 --- a/index.md +++ b/index.md @@ -49,14 +49,18 @@ By the end of this workshop, participants will know how to: * Utilize data volumes at FNAL. * Understand good data management practices. -* Know how to run basic root analysis on Fermilab or CERN unix systems. +* Know how to set up to run basic ROOT-based analysis on Fermilab or CERN unix systems. -There are additional materials provided that explain how to: +There are additional tutorials in this series: -* [Develop configuration files to control batch jobs]({{ site.baseurl }}/07-grid-job-submission) +- [LArSoft Basics](https://dune.github.io/computing-basics-larsoft) +- [Batch submission Basics](https://dune.github.io/computing-basics-batch) + + + You will need to be a DUNE Collaborator (listed member), and have a valid FNAL or CERN computing account to join the tutorial. Contact your DUNE group leader for assistance. From 28d86fbc43f7f59a7d0d2cf585cd8164ad9a57cf Mon Sep 17 00:00:00 2001 From: Heidi Schellman <33669005+hschellman@users.noreply.github.com> Date: Fri, 1 Aug 2025 18:27:04 -0700 Subject: [PATCH 5/5] add in note about rucio config warning message --- _episodes/01.5-documentation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_episodes/01.5-documentation.md b/_episodes/01.5-documentation.md index 1389822..51fa188 100644 --- a/_episodes/01.5-documentation.md +++ b/_episodes/01.5-documentation.md @@ -1,6 +1,6 @@ --- title: DUNE Documentation -teaching: 5 +teaching: 10 exercises: 0 questions: - Where can I find documentation for DUNE