From 1fb501555fd38e43bea516d5dc307c91700f9e2e Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Mon, 2 Nov 2026 07:26:11 +0100 Subject: [PATCH 1/4] doc: design output configuration for JSON output mode Extend the output configuration design to the --json mode of avail, list and spider sub-commands, as a preparatory step to let users select what is reported in JSON documents. Introduce avail_json_output, list_json_output and spider_json_output configuration options, which can be superseded through environment variable or the -o/--output option. Their default value reproduces the JSON document currently produced, so existing consumers are not affected unless configuration is changed. Define a new mtime element, specific to JSON mode, to report the modification time of modulefiles. It is not enabled by default as it requires to stat every modulefile when no module cache file is available. Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Xavier Delaruelle --- .hunspell.en.dic | 2 + doc/source/design/output-configuration.rst | 177 ++++++++++++++++++--- 2 files changed, 160 insertions(+), 19 deletions(-) diff --git a/.hunspell.en.dic b/.hunspell.en.dic index 38599ae9c..720ee12cd 100644 --- a/.hunspell.en.dic +++ b/.hunspell.en.dic @@ -1659,3 +1659,5 @@ misoptval nextargisval otherargv parseModuleCommandArgs +DDTHH +hhmm diff --git a/doc/source/design/output-configuration.rst b/doc/source/design/output-configuration.rst index 39123f592..2a320c9d7 100644 --- a/doc/source/design/output-configuration.rst +++ b/doc/source/design/output-configuration.rst @@ -10,24 +10,29 @@ Configuration - :mconfig:`avail_output`: elements to report on :subcmd:`avail` sub-command regular mode - :mconfig:`avail_terse_output`: elements to report on ``avail`` sub-command terse mode + - :mconfig:`avail_json_output`: elements to report on ``avail`` sub-command JSON mode - :mconfig:`list_output`: elements to report on :subcmd:`list` sub-command regular mode - :mconfig:`list_terse_output`: elements to report on ``list`` sub-command terse mode + - :mconfig:`list_json_output`: elements to report on ``list`` sub-command JSON mode - :mconfig:`spider_output`: elements to report on :subcmd:`spider` sub-command regular mode - :mconfig:`spider_terse_output`: elements to report on ``spider`` sub-command terse mode + - :mconfig:`spider_json_output`: elements to report on ``spider`` sub-command JSON mode Specification ------------- - Output configuration is available - - For ``avail`` sub-command, on regular and :option:`--terse` modes - - For ``list`` sub-command, on regular and :option:`--terse` modes - - For ``spider`` sub-command, on regular and :option:`--terse` modes + - For ``avail`` sub-command, on regular, :option:`--terse` and :option:`--json` modes + - For ``list`` sub-command, on regular, :option:`--terse` and :option:`--json` modes + - For ``spider`` sub-command, on regular, :option:`--terse` and :option:`--json` modes - Output configuration is not available (but could be added later on) - - For :option:`--long` and :option:`--json` modes - - For :subcmd:`aliases`, :subcmd:`savelist`, :subcmd:`whatis`/:subcmd:`search` and :subcmd:`display`/:subcmd:`help` sub-commands + - For :option:`--long` mode + - For :subcmd:`aliases`, :subcmd:`savelist`, :subcmd:`stashlist`, + :subcmd:`whatis`/:subcmd:`search` and :subcmd:`display`/:subcmd:`help` + sub-commands, including their ``--json`` mode - Output configuration defines the content of the output not its format @@ -37,10 +42,13 @@ Specification - ``avail_output``: ``modulepath:alias:dirwsym:sym:tag:variantifspec:key`` - ``avail_terse_output``: ``modulepath:alias:dirwsym:sym:tag:variantifspec`` + - ``avail_json_output``: ``modulepath:alias:dirwsym:sym:tag:via`` - ``list_output``: ``header:idx:variant:sym:tag:key`` - ``list_terse_output``: ``header`` + - ``list_json_output``: ``variant:sym:tag:via`` - ``spider_output``: ``modulepath:alias:dirwsym:sym:tag:variantifspec:via:key`` - ``spider_terse_output``: ``modulepath:alias:dirwsym:sym:tag:variantifspec`` + - ``spider_json_output``: ``modulepath:alias:dirwsym:sym:tag:via`` - The above default value could be superseded: @@ -48,26 +56,34 @@ Specification - :envvar:`MODULES_AVAIL_OUTPUT` to supersede ``avail_output`` default value - :envvar:`MODULES_AVAIL_TERSE_OUTPUT` to supersede ``avail_terse_output`` default value + - :envvar:`MODULES_AVAIL_JSON_OUTPUT` to supersede ``avail_json_output`` default value - :envvar:`MODULES_LIST_OUTPUT` to supersede ``list_output`` default value - :envvar:`MODULES_LIST_TERSE_OUTPUT` to supersede ``list_terse_output`` default value + - :envvar:`MODULES_LIST_JSON_OUTPUT` to supersede ``list_json_output`` default value - :envvar:`MODULES_SPIDER_OUTPUT` to supersede ``spider_output`` default value - :envvar:`MODULES_SPIDER_TERSE_OUTPUT` to supersede ``spider_terse_output`` default value + - :envvar:`MODULES_SPIDER_JSON_OUTPUT` to supersede ``spider_json_output`` default value - with the :option:`-o`/:option:`--output` command-line option - - which applies to the current output mode defined + - which applies to the current output mode defined (regular, terse or JSON) - ``-o``/``--output`` also supersedes environment variable definition - Accepted elements in value lists are: - - For ``avail`` options: modulepath, alias, provided-alias, dirwsym, - indesym, sym, tag, key, variant, variantifspec, hidden, via (only on - regular output) - - For ``list`` options: header, idx, variant, alias, indesym, sym, tag, key, - hidden - - For ``spider`` options: modulepath, alias, provided-alias, dirwsym, - indesym, sym, tag, key, variant, variantifspec, hidden, via (only on - regular output) + - For ``avail`` regular and terse options: modulepath, alias, + provided-alias, dirwsym, indesym, sym, tag, key, variant, variantifspec, + hidden, via (only on regular output) + - For ``avail`` JSON option: modulepath, alias, provided-alias, dirwsym, + sym, tag, variant, variantifspec, hidden, via, mtime + - For ``list`` regular and terse options: header, idx, variant, alias, + indesym, sym, tag, key, hidden + - For ``list`` JSON option: variant, alias, sym, tag, hidden, via, mtime + - For ``spider`` regular and terse options: modulepath, alias, + provided-alias, dirwsym, indesym, sym, tag, key, variant, variantifspec, + hidden, via (only on regular output) + - For ``spider`` JSON option: modulepath, alias, provided-alias, dirwsym, + sym, tag, variant, variantifspec, hidden, via, mtime - If the ``-o``/``--output`` options are wrongly specified @@ -78,7 +94,7 @@ Specification - Option set on unsupported module sub-command - Element set in option value unsupported by module sub-command - Elements set in option value not separated by colon character (``:``) - - Option set on unsupported output format (``--long`` or ``--json``) + - Option set on unsupported output format (``--long``) - The above situations apply whether command is called from the terminal or within a modulefile - For all these new configuration options @@ -87,10 +103,11 @@ Specification - order of elements in the list does not matter - an empty string is a valid value (means only the modulefile name has to be reported) -- If the ``MODULES_AVAIL_OUTPUT``/``MODULES_AVAIL_TERSE_OUTPUT``, - ``MODULES_LIST_OUTPUT``/``MODULES_LIST_TERSE_OUTPUT``, - ``MODULES_SPIDER_OUTPUT``/``MODULES_SPIDER_TERSE_OUTPUT`` env vars are - wrongly specified +- If the ``MODULES_AVAIL_OUTPUT``/``MODULES_AVAIL_TERSE_OUTPUT``/ + ``MODULES_AVAIL_JSON_OUTPUT``, ``MODULES_LIST_OUTPUT``/ + ``MODULES_LIST_TERSE_OUTPUT``/``MODULES_LIST_JSON_OUTPUT``, + ``MODULES_SPIDER_OUTPUT``/``MODULES_SPIDER_TERSE_OUTPUT``/ + ``MODULES_SPIDER_JSON_OUTPUT`` env vars are wrongly specified - Their value is ignored - So the default value takes precedence, unless a ``-o``/``--output`` option is specified @@ -119,6 +136,128 @@ Specification - a module appearing in several modulepaths is only reported once - tags or symbols applying to a lower priority module with same name are still reported +JSON output +^^^^^^^^^^^ + +- Rationale + + - Each element in the JSON document is a cost: *variant* requires a scan + evaluation of every modulefile, *mtime* requires to stat every modulefile + when no module cache file is available + - Default content is thus kept minimal to preserve response time and an + output configuration lets user or site add the elements they need + - Output configuration is applied to ``--json`` mode through the same + mechanisms as regular and terse modes (configuration option, environment + variable and ``-o``/``--output`` command-line option) rather than through + dedicated options + +- The content of the JSON document is defined by the ``_json_output`` + configuration option relative to the sub-command run + + - Elements in these options select the JSON keys reported in each module + object + - The ``-o``/``--output`` option applies to ``_json_output`` when + ``--json`` mode is set, ``+`` and ``-`` prefixes are supported the same + way as on regular and terse modes + - ``-o``/``--output`` is still refused on ``--long`` mode + +- Some keys are intrinsic to the JSON objects and are always reported + + - ``name`` and ``type`` keys on every object + - ``pathname`` key on ``modulefile`` objects + - ``target`` key on ``alias`` objects + - When output configuration value is an empty string, module objects only + hold these keys + +- Mapping between elements and the content of the JSON document + + - *modulepath*: top-level JSON object is keyed by modulepath, each value + being an object keyed by module name (``avail`` and ``spider`` only) + - *alias*: objects of type ``alias`` are reported + - *provided-alias*: like *alias* plus the aliases provided by modulefiles, + which requires a scan evaluation of every modulefile (``avail`` and + ``spider`` only) + - *dirwsym*: objects of type ``directory`` are reported for directories + associated with symbolic versions (``avail`` and ``spider`` only) + - *sym*: ``symbols`` key is added to ``modulefile``, ``alias`` and + ``directory`` objects + - *tag*: ``tags`` key is added to ``modulefile`` and ``alias`` objects + - *variant*: ``variants`` key is added to ``modulefile`` objects + + - Variants and their possible values on ``avail`` and ``spider`` + - Variant values selected for loaded modules on ``list`` + + - *variantifspec*: like *variant* but only if a variant has been specified + in search query (``avail`` and ``spider`` only) + - *hidden*: hidden modules are reported, like on regular output mode + - *via*: ``via`` key is added to every object + - *mtime*: ``mtime`` key is added to ``modulefile`` objects + +- *mtime* is a new element specific to JSON output mode + + - It reports the modification time of the modulefile, like ``--long`` mode + does on text output + - Value is a JSON string in ISO 8601 format with local time and UTC offset + (``YYYY-MM-DDTHH:MM:SS±hhmm``) to be both human-readable and parsable by + JSON consumers without ambiguity, unlike ``YYYY/MM/DD HH:MM:SS`` format + used on ``--long`` mode + - No ``mtime`` key is set on ``alias`` and ``directory`` objects, like + ``--long`` mode which reports no modification time for them + - Modification time of virtual modules is the one of the modulefile they + point to + - Modification time is only fetched when *mtime* element is enabled, as it + requires to stat every modulefile when no module cache file is available + (modification time is recorded in module cache files) + - *mtime* is not accepted on regular and terse output modes as ``--long`` + mode already serves this purpose there + +- Elements not accepted on JSON output mode + + - *key*: no legend is reported on JSON output + - *header* and *idx* (``list``): JSON document has no introduction sentence + and the position of a loaded module is its position in the top-level + object + - *indesym*: symbolic versions are already reported in the ``symbols`` key + of the object they are attached to + +- When an element is not enabled, its corresponding key is omitted from the + JSON objects rather than set to an empty value + + - A JSON consumer can tell from the document which elements were reported + - Document is smaller when few elements are requested + +- When *modulepath* element is removed from an ``avail`` or ``spider`` JSON + option + + - Top-level JSON object is directly keyed by module name, like the ``list`` + sub-command JSON document + - Modules from the different modulepaths are merged as a single set, with + the same rules as regular output mode (a module appearing in several + modulepaths is only reported once) + - Aliases defined in global/user rc are merged in this single set too + +- Default values of ``avail_json_output``, ``list_json_output`` and + ``spider_json_output`` reproduce the JSON document produced prior to the + introduction of these options + + - Existing JSON consumers get the exact same document unless configuration + is changed + - *variant* is thus part of ``list_json_output`` default value but not of + ``avail_json_output`` and ``spider_json_output`` default values + - *via* is part of ``list_json_output`` default value even if its value is + currently always empty on ``list`` sub-command (see + :ref:`spider-sub-command` design notes) + +- ``--all`` option still unveils hidden modules on JSON output mode, like + *hidden* element does + +- Install-time options are added to change the default value of these + configuration options + + - ``--with-avail-json-output`` + - ``--with-list-json-output`` + - ``--with-spider-json-output`` + Output key ^^^^^^^^^^ From fcc0d42bc6e86b8682bf2d311ab8619ee17d8f8d Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Mon, 2 Nov 2026 08:12:34 +0100 Subject: [PATCH 2/4] Add output configuration for JSON output mode Introduce avail_json_output, list_json_output and spider_json_output configuration options to define the content of the JSON output of avail, list and spider sub-commands. Elements in these options select the keys reported in each module JSON object. The -o/--output command line switch is now accepted on --json mode to supersede them, with support of the + and - prefixes. Default value of these options can be set at installation time with the --with-avail-json-output, --with-list-json-output and --with-spider-json-output configure options. Default values reproduce the JSON document produced so far, so existing JSON consumers are not affected unless configuration is changed. When an element is not enabled, its key is omitted from JSON objects. When modulepath element is removed, the top-level JSON object is directly keyed by module names. Variants are now reported on avail and spider JSON output when variant or variantifspec element is enabled: the possible values of each variant are set as a JSON array of strings. Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Xavier Delaruelle --- INSTALL.rst | 65 ++++++ Makefile | 3 + Makefile.inc.in | 3 + NEWS.rst | 10 + configure | 37 +++- doc/source/changes.rst | 17 ++ doc/source/design/output-configuration.rst | 6 +- doc/source/module.rst | 199 +++++++++++++++++- init/Makefile | 2 +- init/fish_completion | 2 +- init/zsh-functions/_module.in | 2 +- site.exp.in | 3 + tcl/init.tcl.in | 8 + tcl/main.tcl.in | 2 +- tcl/modfind.tcl.in | 10 +- tcl/report.tcl.in | 39 ++-- testsuite/install.00-init/010-environ.exp | 3 + testsuite/modules.00-init/010-environ.exp | 3 + .../020-avail_output-variant.exp | 17 +- testsuite/modules.70-maint/220-config.exp | 18 ++ .../modules.70-maint/360-avail_output.exp | 126 ++++++++++- .../modules.70-maint/361-list_output.exp | 74 ++++++- .../modules.70-maint/362-spider_output.exp | 65 +++++- 23 files changed, 665 insertions(+), 49 deletions(-) diff --git a/INSTALL.rst b/INSTALL.rst index df633bcc8..3d55a7ee7 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -990,6 +990,24 @@ instance :instopt:`--without-modulepath<--with-modulepath>`): .. versionadded:: 5.4 +.. instopt:: --with-avail-json-output=LIST + + Specify the content to report on avail sub-command JSON output in addition + to the available module names. Elements accepted in LIST are: + ``modulepath``, ``alias``, ``provided-alias``, ``dirwsym``, ``sym``, + ``tag``, ``hidden``, ``variant``, ``variantifspec`` and ``via`` (elements in + LIST are separated by ``:``). The order of the elements in LIST + does not matter. + (default=\ ``modulepath:alias:dirwsym:sym:tag:via``) + + This installation option defines the default value of the + :mconfig:`avail_json_output` configuration option which could be changed + after installation with the :subcmd:`config` sub-command. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. instopt:: --with-avail-output=LIST Specify the content to report on avail sub-command regular output in addition @@ -1257,6 +1275,23 @@ instance :instopt:`--without-modulepath<--with-modulepath>`): .. versionchanged:: 5.6 Default ``nearly-forbidden`` tag color changed +.. instopt:: --with-list-json-output=LIST + + Specify the content to report on list sub-command JSON output in addition + to the loaded module names. Elements accepted in LIST are: + ``variant``, ``alias``, ``sym``, ``tag``, ``hidden`` and ``via`` (elements + in LIST are separated by ``:``). The order of the elements in LIST + does not matter. + (default=\ ``variant:sym:tag:via``) + + This installation option defines the default value of the + :mconfig:`list_json_output` configuration option which could be changed + after installation with the :subcmd:`config` sub-command. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. instopt:: --with-list-output=LIST Specify the content to report on list sub-command regular output in addition @@ -1504,6 +1539,24 @@ instance :instopt:`--without-modulepath<--with-modulepath>`): .. versionadded:: 4.3 +.. instopt:: --with-spider-json-output=LIST + + Specify the content to report on spider sub-command JSON output in addition + to the available module names. Elements accepted in LIST are: + ``modulepath``, ``alias``, ``provided-alias``, ``dirwsym``, ``sym``, + ``tag``, ``hidden``, ``variant``, ``variantifspec`` and ``via`` (elements in + LIST are separated by ``:``). The order of the elements in LIST + does not matter. + (default=\ ``modulepath:alias:dirwsym:sym:tag:via``) + + This installation option defines the default value of the + :mconfig:`spider_json_output` configuration option which could be changed + after installation with the :subcmd:`config` sub-command. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. instopt:: --with-spider-output=LIST Specify the content to report on spider sub-command regular output in @@ -1760,6 +1813,10 @@ installation. | | | :envvar:`MODULES_AVAIL_INDEPTH`, | | | | | | :option:`--indepth`, :option:`--no-indepth` | | | +-----------------------------------+----------------------------------------------+----------------------------------------------+--------------+-----------+ +| :mconfig:`avail_json_output` | modulepath:alias:dirwsym:sym:tag:via | :instopt:`--with-avail-json-output`, | | | +| | | :envvar:`MODULES_AVAIL_JSON_OUTPUT`, | | | +| | | :option:`--output` | | | ++-----------------------------------+----------------------------------------------+----------------------------------------------+--------------+-----------+ | :mconfig:`avail_output` | modulepath:alias:dirwsym:sym:tag:\ | :instopt:`--with-avail-output`, | | | | | variantifspec:key | :envvar:`MODULES_AVAIL_OUTPUT`, | | | | | | :option:`--output` | | | @@ -1837,6 +1894,10 @@ installation. +-----------------------------------+----------------------------------------------+----------------------------------------------+--------------+-----------+ | :mconfig:`linked_envvars` | *Empty by default* | :envvar:`MODULES_LINKED_ENVVARS` | | | +-----------------------------------+----------------------------------------------+----------------------------------------------+--------------+-----------+ +| :mconfig:`list_json_output` | variant:sym:tag:via | :instopt:`--with-list-json-output`, | | | +| | | :envvar:`MODULES_LIST_JSON_OUTPUT`, | | | +| | | :option:`--output` | | | ++-----------------------------------+----------------------------------------------+----------------------------------------------+--------------+-----------+ | :mconfig:`list_output` | ``header:idx:variant:sym:tag:key`` | :instopt:`--with-list-output`, | | | | | | :envvar:`MODULES_LIST_OUTPUT`, | | | | | | :option:`--output` | | | @@ -1923,6 +1984,10 @@ installation. | | | :envvar:`MODULES_SPIDER_INDEPTH`, | | | | | | :option:`--indepth`, :option:`--no-indepth` | | | +-----------------------------------+----------------------------------------------+----------------------------------------------+--------------+-----------+ +| :mconfig:`spider_json_output` | modulepath:alias:dirwsym:sym:tag:via | :instopt:`--with-spider-json-output`, | | | +| | | :envvar:`MODULES_SPIDER_JSON_OUTPUT`, | | | +| | | :option:`--output` | | | ++-----------------------------------+----------------------------------------------+----------------------------------------------+--------------+-----------+ | :mconfig:`spider_output` | modulepath:alias:dirwsym:sym:tag:\ | :instopt:`--with-spider-output`, | | | | | variantifspec:via:key | :envvar:`MODULES_SPIDER_OUTPUT`, | | | | | | :option:`--output` | | | diff --git a/Makefile b/Makefile index 692489eec..b332eaefb 100644 --- a/Makefile +++ b/Makefile @@ -497,6 +497,9 @@ sed -e 's|@prefix@|$(prefix)|g' \ -e 's|@listterseoutput@|$(listterseoutput)|g' \ -e 's|@spideroutput@|$(spideroutput)|g' \ -e 's|@spiderterseoutput@|$(spiderterseoutput)|g' \ + -e 's|@availjsonoutput@|$(availjsonoutput)|g' \ + -e 's|@listjsonoutput@|$(listjsonoutput)|g' \ + -e 's|@spiderjsonoutput@|$(spiderjsonoutput)|g' \ -e 's|@variantshortcut@|$(variantshortcut)|g' \ -e 's|@editor@|$(editor)|g' \ -e 's|@autohandling@|$(setautohandling)|g' \ diff --git a/Makefile.inc.in b/Makefile.inc.in index 4b4e5a6c7..8f7537bd9 100644 --- a/Makefile.inc.in +++ b/Makefile.inc.in @@ -160,6 +160,9 @@ listoutput := @listoutput@ listterseoutput := @listterseoutput@ spideroutput := @spideroutput@ spiderterseoutput := @spiderterseoutput@ +availjsonoutput := @availjsonoutput@ +listjsonoutput := @listjsonoutput@ +spiderjsonoutput := @spiderjsonoutput@ # variant configuration variantshortcut := @variantshortcut@ diff --git a/NEWS.rst b/NEWS.rst index d53d08b1c..90df4ecd3 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -38,6 +38,16 @@ Modules 5.8.0 (not yet released) concurrently, each group of files being run in a private copy of the testsuite directory. Coverage mode (``COVERAGE=y``) is supported, the coverage logs of each group of files are merged at the end of the run. +* Add :mconfig:`avail_json_output`, :mconfig:`list_json_output` and + :mconfig:`spider_json_output` configuration options to define the content + of the JSON output of :subcmd:`avail`, :subcmd:`list` and :subcmd:`spider` + sub-commands. Default value of these options can be set at installation + time with :instopt:`--with-avail-json-output`, + :instopt:`--with-list-json-output` and :instopt:`--with-spider-json-output` + options and superseded with the :envvar:`MODULES_AVAIL_JSON_OUTPUT`, + :envvar:`MODULES_LIST_JSON_OUTPUT` and :envvar:`MODULES_SPIDER_JSON_OUTPUT` + environment variables or the :option:`--output` command line switch, which + is now accepted on :option:`--json` output mode. .. _5.7 release notes: diff --git a/configure b/configure index ab465f834..13ae2de98 100755 --- a/configure +++ b/configure @@ -37,7 +37,8 @@ darkbgcolors lightbgcolors termbg lockedconfigs icase unloadmatchorder \ searchmatch modulepath loadedmodules quarantinevars wa277 pathentryreorder advversspec ml \ windowssupport nearlyforbiddendays implicitrequirement tagabbrev \ tagcolorname mcookieversioncheck availoutput availterseoutput listoutput \ -listterseoutput editor variantshortcut bashcompletiondir fishcompletiondir \ +listterseoutput availjsonoutput listjsonoutput spiderjsonoutput editor \ +variantshortcut bashcompletiondir fishcompletiondir \ zshcompletiondir tcllinter tcllinteropts nagelfardatadir nagelfaraddons \ stickypurge uniquenameloaded abortonerror sourcecache logger loggeropts \ loggedevents conflictunload spideroutput spiderterseoutput spiderindepth \ @@ -152,6 +153,9 @@ listoutput='header:idx:variant:sym:tag:key' listterseoutput='header' spideroutput='modulepath:alias:dirwsym:sym:tag:variantifspec:via:key' spiderterseoutput=$availterseoutput +availjsonoutput='modulepath:alias:dirwsym:sym:tag:via' +listjsonoutput='variant:sym:tag:via' +spiderjsonoutput=$availjsonoutput editor='vi' variantshortcut= bashcompletiondir= @@ -364,6 +368,14 @@ Optional Packages: \`reload', \`switch', \`switch_unload', \`try-load' and \`unload' (elements in LIST are separated by \`:') [$abortonerror] + --with-avail-json-output=LIST + specify content to report on avail sub-command + JSON output. elements accepted in LIST are: + \`modulepath', \`alias', \`provided-alias', + \`dirwsym', \`sym', \`tag', \`hidden', \`variant', + \`variantifspec' and \`via' (elements in LIST are + separated by \`:') + [$availjsonoutput] --with-avail-output=LIST specify content to report on avail sub-command regular output. elements accepted in LIST are: @@ -423,6 +435,12 @@ Optional Packages: color is defined to \`light'. Expect the same syntax than described for \`--with-dark-background-colors' [$lightbgcolors] + --with-list-json-output=LIST + specify content to report on list sub-command + JSON output. elements accepted in LIST are: + \`variant', \`alias', \`sym', \`tag', \`hidden' and + \`via' (elements in LIST are separated by \`:') + [$listjsonoutput] --with-list-output=LIST specify content to report on list sub-command regular output. elements accepted in LIST are: @@ -489,6 +507,14 @@ Optional Packages: match the beginning of module name (\`starts_with') or any part of module name (\`contains') [$searchmatch] + --with-spider-json-output=LIST + specify content to report on spider sub-command + JSON output. elements accepted in LIST are: + \`modulepath', \`alias', \`provided-alias', + \`dirwsym', \`sym', \`tag', \`hidden', \`variant', + \`variantifspec' and \`via' (elements in LIST are + separated by \`:') + [$spiderjsonoutput] --with-spider-output=LIST specify content to report on spider sub-command regular output. elements accepted in LIST are: @@ -972,6 +998,15 @@ for arg in "$@"; do --with-spider-terse-output=*|--without-spider-terse-output) allowedval=" modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec " ; check_and_get_package_value_list "spiderterseoutput" "$arg" "$allowedval" "" ;; + --with-avail-json-output=*|--without-avail-json-output) + allowedval=" modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via " ; + check_and_get_package_value_list "availjsonoutput" "$arg" "$allowedval" "" ;; + --with-list-json-output=*|--without-list-json-output) + allowedval=" variant alias sym tag hidden via " ; + check_and_get_package_value_list "listjsonoutput" "$arg" "$allowedval" "" ;; + --with-spider-json-output=*|--without-spider-json-output) + allowedval=" modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via " ; + check_and_get_package_value_list "spiderjsonoutput" "$arg" "$allowedval" "" ;; --with-variant-shortcut=*|--without-variant-shortcut) variantshortcut=$(get_package_value "$arg" "") ;; --with-editor=*|--without-editor) diff --git a/doc/source/changes.rst b/doc/source/changes.rst index d344c4b7b..f17fef269 100644 --- a/doc/source/changes.rst +++ b/doc/source/changes.rst @@ -887,6 +887,9 @@ The following command line switches appeared on Modules 5. Starting Modules 5.6, :subcmd:`spider` sub-command is supported. + Starting Modules 5.8, this command line switch is accepted on JSON output + mode. + :option:`--force` Starting Modules 5.4, this command line switch by-passes *abort on error* @@ -914,6 +917,13 @@ The following command line switches appeared on Modules 5. Starting Modules 5.6, :subcmd:`spider` sub-command is supported. +:option:`--json` + + Starting Modules 5.8, content of :subcmd:`avail`, :subcmd:`list` and + :subcmd:`spider` JSON output is defined by the :mconfig:`avail_json_output`, + :mconfig:`list_json_output` and :mconfig:`spider_json_output` configuration + options and the :option:`--output` command line switch is accepted. + :option:`--terse` Starting Modules 5.8, :subcmd:`search` and :subcmd:`whatis` sub-commands are @@ -1133,6 +1143,10 @@ The following environment variables appeared on Modules 5. | | :envvar:`MODULES_MODULEPATH_IGNORE`, | | | :envvar:`MODULES_IGNORE_CACHE_SUBCMDS` | +------------+-----------------------------------------------------------------+ +| 5.8 | :envvar:`MODULES_AVAIL_JSON_OUTPUT`, | +| | :envvar:`MODULES_LIST_JSON_OUTPUT`, | +| | :envvar:`MODULES_SPIDER_JSON_OUTPUT` | ++------------+-----------------------------------------------------------------+ Modules Specific Tcl Commands """"""""""""""""""""""""""""" @@ -1355,6 +1369,9 @@ The following Modules configuration option has been introduced on Modules 5. | | :mconfig:`domainname`, :mconfig:`modulepath_ignore`, | | | :mconfig:`ignore_cache_subcmds` | +------------+-----------------------------------------------------------------+ +| 5.8 | :mconfig:`avail_json_output`, :mconfig:`list_json_output`, | +| | :mconfig:`spider_json_output` | ++------------+-----------------------------------------------------------------+ :mconfig:`auto_handling` diff --git a/doc/source/design/output-configuration.rst b/doc/source/design/output-configuration.rst index 2a320c9d7..e06b069f8 100644 --- a/doc/source/design/output-configuration.rst +++ b/doc/source/design/output-configuration.rst @@ -184,7 +184,9 @@ JSON output - *tag*: ``tags`` key is added to ``modulefile`` and ``alias`` objects - *variant*: ``variants`` key is added to ``modulefile`` objects - - Variants and their possible values on ``avail`` and ``spider`` + - Variants and their possible values on ``avail`` and ``spider``, each + value list being a JSON array of strings (``on`` and ``off`` for a + Boolean variant, ``*`` denoting a free value variant) - Variant values selected for loaded modules on ``list`` - *variantifspec*: like *variant* but only if a variant has been specified @@ -203,6 +205,8 @@ JSON output used on ``--long`` mode - No ``mtime`` key is set on ``alias`` and ``directory`` objects, like ``--long`` mode which reports no modification time for them + - Value is ``null`` if modification time cannot be fetched, for instance + when the modulefile of a loaded module does not exist anymore - Modification time of virtual modules is the one of the modulefile they point to - Modification time is only fetched when *mtime* element is enabled, as it diff --git a/doc/source/module.rst b/doc/source/module.rst index 9d452f1a7..bd24c50ab 100644 --- a/doc/source/module.rst +++ b/doc/source/module.rst @@ -422,6 +422,11 @@ switches are accepted: :subcmd:`search`, :subcmd:`spider`, :subcmd:`stashlist` and :subcmd:`whatis` output in JSON format. + Content of :subcmd:`avail`, :subcmd:`list` and :subcmd:`spider` JSON output + is defined by the :mconfig:`avail_json_output`, :mconfig:`list_json_output` + and :mconfig:`spider_json_output` configuration options, which can be + overridden with the :option:`--output`/:option:`-o` command line switches. + .. only:: html or latex .. versionadded:: 4.5 @@ -432,6 +437,10 @@ switches are accepted: .. versionchanged:: 5.6 Support for :subcmd:`spider` sub-command added + .. versionchanged:: 5.8 + Options :option:`--output`/:option:`-o` supported on :subcmd:`avail`, + :subcmd:`list` and :subcmd:`spider` sub-commands + .. option:: --latest, -L @@ -525,9 +534,9 @@ switches are accepted: Define the content to report in addition to module names. This option is supported by :subcmd:`avail`, :subcmd:`list` and :subcmd:`spider` - sub-commands on their regular or terse output modes. Accepted values are a - *LIST* of elements to report separated by colon character (``:``). The order - of the elements in *LIST* does not matter. + sub-commands on their regular, terse or JSON output modes. Accepted values + are a *LIST* of elements to report separated by colon character (``:``). The + order of the elements in *LIST* does not matter. Accepted elements in *LIST* for :subcmd:`avail` and :subcmd:`spider` sub-command are: *modulepath*, *alias*, *provided-alias*, *dirwsym*, @@ -537,14 +546,21 @@ switches are accepted: Accepted elements in *LIST* for :subcmd:`list` sub-command are: *header*, *idx*, *variant*, *alias*, *indesym*, *sym*, *tag*, *hidden* and *key*. + On JSON output mode, accepted elements in *LIST* for :subcmd:`avail` and + :subcmd:`spider` sub-command are: *modulepath*, *alias*, *provided-alias*, + *dirwsym*, *sym*, *tag*, *hidden*, *variant*, *variantifspec* and *via*. + Accepted elements for :subcmd:`list` sub-command are: *variant*, *alias*, + *sym*, *tag*, *hidden* and *via*. + The order of the elements in *LIST* does not matter. Module names are the only content reported when *LIST* is set to an empty value. *LIST* may be prefixed by ``+`` or ``-`` character to indicate respectively to append it to or subtract it from current configuration option value. - See also :envvar:`MODULES_AVAIL_OUTPUT`, :envvar:`MODULES_LIST_OUTPUT` and - :envvar:`MODULES_SPIDER_OUTPUT`. + See also :envvar:`MODULES_AVAIL_OUTPUT`, :envvar:`MODULES_LIST_OUTPUT`, + :envvar:`MODULES_SPIDER_OUTPUT`, :envvar:`MODULES_AVAIL_JSON_OUTPUT`, + :envvar:`MODULES_LIST_JSON_OUTPUT` and :envvar:`MODULES_SPIDER_JSON_OUTPUT`. .. only:: html or latex @@ -577,6 +593,9 @@ switches are accepted: Elements *provided-alias* and *via* added for :subcmd:`avail` and :subcmd:`spider` sub-commands + .. versionchanged:: 5.8 + JSON output mode supported + .. option:: --paginate, -p Pipe all message output into :command:`less` (or if set, to the command @@ -860,6 +879,11 @@ Module Sub-Commands Results from a multi *pattern* search are consolidated under a single output + .. versionchanged:: 5.8 + Option :option:`--output`/:option:`-o` compatible with JSON output + mode + + .. subcmd:: cachebuild [modulepath...] Build module cache file for designated modulepaths. If no argument is @@ -984,6 +1008,23 @@ Module Sub-Commands command line switches change the value of this configuration option. See :envvar:`MODULES_AVAIL_INDEPTH` description for details. + .. mconfig:: avail_json_output + + Content to report in addition to module names on :subcmd:`avail` sub-command + JSON output mode. + + Default value is ``modulepath:alias:dirwsym:sym:tag:via``. It can be + changed at installation time with :instopt:`--with-avail-json-output` + option. The :envvar:`MODULES_AVAIL_JSON_OUTPUT` environment variable is + defined by :subcmd:`config` sub-command when changing this configuration + option from its default value. The :option:`--output`/:option:`-o` command + line switches change the value of this configuration option. See + :envvar:`MODULES_AVAIL_JSON_OUTPUT` description for details. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. mconfig:: avail_output Content to report in addition to module names on :subcmd:`avail` sub-command @@ -1342,6 +1383,23 @@ Module Sub-Commands .. versionadded:: 5.7 + .. mconfig:: list_json_output + + Content to report in addition to module names on :subcmd:`list` sub-command + JSON output mode. + + Default value is ``variant:sym:tag:via``. It can be + changed at installation time with :instopt:`--with-list-json-output` + option. The :envvar:`MODULES_LIST_JSON_OUTPUT` environment variable is + defined by :subcmd:`config` sub-command when changing this configuration + option from its default value. The :option:`--output`/:option:`-o` command + line switches change the value of this configuration option. See + :envvar:`MODULES_LIST_JSON_OUTPUT` description for details. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. mconfig:: list_output Content to report in addition to module names on :subcmd:`list` sub-command @@ -1738,6 +1796,23 @@ Module Sub-Commands .. versionadded:: 5.6 + .. mconfig:: spider_json_output + + Content to report in addition to module names on :subcmd:`spider` sub-command + JSON output mode. + + Default value is ``modulepath:alias:dirwsym:sym:tag:via``. It can be + changed at installation time with :instopt:`--with-spider-json-output` + option. The :envvar:`MODULES_SPIDER_JSON_OUTPUT` environment variable is + defined by :subcmd:`config` sub-command when changing this configuration + option from its default value. The :option:`--output`/:option:`-o` command + line switches change the value of this configuration option. See + :envvar:`MODULES_SPIDER_JSON_OUTPUT` description for details. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. mconfig:: spider_output Content to report in addition to module names on :subcmd:`spider` @@ -2208,6 +2283,11 @@ Module Sub-Commands .. versionchanged:: 5.3 *pattern* may include variant specification to filter results + .. versionchanged:: 5.8 + Option :option:`--output`/:option:`-o` compatible with JSON output + mode + + .. subcmd:: load [options] modulefile... Load *modulefile* into the shell environment. @@ -2859,6 +2939,11 @@ Module Sub-Commands .. versionadded:: 5.6 + .. versionchanged:: 5.8 + Option :option:`--output`/:option:`-o` compatible with JSON output + mode + + .. subcmd:: stash [-f] :subcmd:`Save` current environment in a stash collection then @@ -3701,9 +3786,14 @@ When a shortcut character is defined for a variant (see shortcut for *variant1*: ``foo/1.2{%value:+variant2}``. When the JSON output mode is enabled (with :option:`--json`), variants are -reported under the ``variants`` JSON object as name/value pairs. Values of -Boolean variant are set as JSON Boolean. Other values are set as JSON strings. -Variant shortcut and color rendering do not apply on JSON output. +reported under the ``variants`` JSON object as name/value pairs if the +*variant* element is part of the JSON output configuration (see +:envvar:`MODULES_AVAIL_JSON_OUTPUT` and :envvar:`MODULES_LIST_JSON_OUTPUT`). +On :subcmd:`list` sub-command, values of Boolean variant are set as JSON +Boolean and other values are set as JSON strings. On :subcmd:`avail` and +:subcmd:`spider` sub-commands, the possible values of each variant are set as +a JSON array of strings. Variant shortcut and color rendering do not apply on +JSON output. .. only:: html or latex @@ -4973,6 +5063,48 @@ ENVIRONMENT .. versionadded:: 4.3 +.. envvar:: MODULES_AVAIL_JSON_OUTPUT + + A colon separated list of the elements to report in addition to module names + on :subcmd:`avail` sub-command JSON output mode. + + Accepted elements that can be set in value list are: + + * ``alias``: module aliases. + * ``provided-alias``: show module aliases and evaluate all modulefiles to get + aliases provided by them. + * ``dirwsym``: directories associated with symbolic versions. + * ``hidden``: show all hidden modules. + * ``modulepath``: available modules grouped by modulepath, each modulepath + name being a key of the top-level JSON object. + * ``sym``: ``symbols`` key set on each JSON object, with the symbolic + versions associated with available module as value. + * ``tag``: ``tags`` key set on each JSON object, with the tags associated + with available module as value. + * ``variant``: ``variants`` key set on each modulefile JSON object, with + the variants and their possible values as value. + * ``variantifspec``: like ``variant`` but only if a variant has been + specified in search query. + * ``via``: ``via`` key set on each JSON object, with the loaded modules that + enable the modulepath of available module as value. + + The order of the elements in the list does not matter. Only the ``name``, + ``type``, ``pathname`` and ``target`` keys are reported on JSON objects when + *LIST* is set to an empty value. + + In case the ``modulepath`` element is missing from value list, the available + modules from global/user rc and all enabled modulepaths are reported as a + single list, each module name being a key of the top-level JSON object. + + This environment variable value supersedes the default value set in the + :mconfig:`avail_json_output` configuration option. It can be defined with + the :subcmd:`config` sub-command. The :option:`--output`/:option:`-o` command + line switches override this environment variable. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. envvar:: MODULES_AVAIL_OUTPUT A colon separated list of the elements to report in addition to module names @@ -5542,6 +5674,36 @@ ENVIRONMENT .. versionadded:: 5.7 +.. envvar:: MODULES_LIST_JSON_OUTPUT + + A colon separated list of the elements to report in addition to module names + on :subcmd:`list` sub-command JSON output mode. + + Accepted elements that can be set in value list are: + + * ``alias``: module aliases targeting loaded modules. + * ``hidden``: show hidden loaded modules. + * ``sym``: ``symbols`` key set on each JSON object, with the symbolic + versions associated with loaded module as value. + * ``tag``: ``tags`` key set on each JSON object, with the tags associated + with loaded module as value. + * ``variant``: ``variants`` key set on each modulefile JSON object, with + the variant values selected for loaded module as value. + * ``via``: ``via`` key set on each JSON object, currently always empty. + + The order of the elements in the list does not matter. Only the ``name``, + ``type``, ``pathname`` and ``target`` keys are reported on JSON objects when + *LIST* is set to an empty value. + + This environment variable value supersedes the default value set in the + :mconfig:`list_json_output` configuration option. It can be defined with the + :subcmd:`config` sub-command. The :option:`--output`/:option:`-o` command + line switches override this environment variable. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. envvar:: MODULES_LIST_OUTPUT A colon separated list of the elements to report in addition to module names @@ -6116,6 +6278,27 @@ ENVIRONMENT .. versionadded:: 5.6 +.. envvar:: MODULES_SPIDER_JSON_OUTPUT + + A colon separated list of the elements to report in addition to module names + on :subcmd:`spider` sub-command JSON output mode. + + See :envvar:`MODULES_AVAIL_JSON_OUTPUT` to get the accepted elements that + can be set in value list. + + The order of the elements in the list does not matter. Only the ``name``, + ``type``, ``pathname`` and ``target`` keys are reported on JSON objects when + *LIST* is set to an empty value. + + This environment variable value supersedes the default value set in the + :mconfig:`spider_json_output` configuration option. It can be defined with + the :subcmd:`config` sub-command. The :option:`--output`/:option:`-o` + command line switches override this environment variable. + + .. only:: html or latex + + .. versionadded:: 5.8 + .. envvar:: MODULES_SPIDER_OUTPUT A colon separated list of the elements to report in addition to module names diff --git a/init/Makefile b/init/Makefile index 65c41c5a0..e97ee9c20 100644 --- a/init/Makefile +++ b/init/Makefile @@ -153,7 +153,7 @@ comp_stashpop_opts := --force -f comp_source_opts := --auto --no-auto --force -f comp_path_opts := -d --delim --duplicates comp_rm_path_opts := -d --delim --index -comp_config_opts := --dump-state --reset abort_on_error advanced_version_spec auto_handling avail_indepth avail_output avail_terse_output cache_buffer_bytes cache_expiry_secs collection_pin_version collection_pin_tag collection_target color colors conflict_unload contact csh_limit domainname editor extended_default extra_siteconfig hide_auto_loaded home icase ignore_cache ignore_cache_subcmds ignore_user_rc ignored_dirs implicit_default implicit_requirement init_envvars info_extension linked_envvars list_output list_terse_output locked_configs logged_events logger mcookie_check mcookie_version_check ml modulepath_ignore nearly_forbidden_days non_exportable_tags pager paginate path_entry_reorder protected_envvars quarantine_support rcfile redirect_output require_via reset_target_state run_quarantine search_match set_shell_startup shells_with_ksh_fpath silent_shell_debug siteconfig source_cache spider_indepth spider_output spider_terse_output sticky_purge tag_abbrev tag_color_name tcl_ext_lib tcl_linter term_background term_width unique_name_loaded unload_match_order variant_shortcut verbosity wa_277 +comp_config_opts := --dump-state --reset abort_on_error advanced_version_spec auto_handling avail_indepth avail_json_output avail_output avail_terse_output cache_buffer_bytes cache_expiry_secs collection_pin_version collection_pin_tag collection_target color colors conflict_unload contact csh_limit domainname editor extended_default extra_siteconfig hide_auto_loaded home icase ignore_cache ignore_cache_subcmds ignore_user_rc ignored_dirs implicit_default implicit_requirement init_envvars info_extension linked_envvars list_json_output list_output list_terse_output locked_configs logged_events logger mcookie_check mcookie_version_check ml modulepath_ignore nearly_forbidden_days non_exportable_tags pager paginate path_entry_reorder protected_envvars quarantine_support rcfile redirect_output require_via reset_target_state run_quarantine search_match set_shell_startup shells_with_ksh_fpath silent_shell_debug siteconfig source_cache spider_indepth spider_json_output spider_output spider_terse_output sticky_purge tag_abbrev tag_color_name tcl_ext_lib tcl_linter term_background term_width unique_name_loaded unload_match_order variant_shortcut verbosity wa_277 define translate-in-script $(ECHO_GEN) diff --git a/init/fish_completion b/init/fish_completion index 6a12b5544..297438a86 100644 --- a/init/fish_completion +++ b/init/fish_completion @@ -76,7 +76,7 @@ complete -c module -n '__fish_module_use_stashlist' -f -a "(module stashlist --c /Stash collection list\$/d; \ /:\$/d; \ /:ERROR:/d;')" -complete -c module -n '__fish_module_use_config' -f -a "--dump-state --reset abort_on_error advanced_version_spec auto_handling avail_indepth avail_output avail_terse_output cache_buffer_bytes cache_expiry_secs collection_pin_version collection_pin_tag collection_target color colors conflict_unload contact csh_limit domainname editor extended_default extra_siteconfig hide_auto_loaded home icase ignore_cache ignore_cache_subcmds ignore_user_rc ignored_dirs implicit_default implicit_requirement init_envvars info_extension linked_envvars list_output list_terse_output locked_configs logged_events logger mcookie_check mcookie_version_check ml modulepath_ignore nearly_forbidden_days non_exportable_tags pager paginate path_entry_reorder protected_envvars quarantine_support rcfile redirect_output require_via reset_target_state run_quarantine search_match set_shell_startup shells_with_ksh_fpath silent_shell_debug siteconfig source_cache spider_indepth spider_output spider_terse_output sticky_purge tag_abbrev tag_color_name tcl_ext_lib tcl_linter term_background term_width unique_name_loaded unload_match_order variant_shortcut verbosity wa_277" +complete -c module -n '__fish_module_use_config' -f -a "--dump-state --reset abort_on_error advanced_version_spec auto_handling avail_indepth avail_json_output avail_output avail_terse_output cache_buffer_bytes cache_expiry_secs collection_pin_version collection_pin_tag collection_target color colors conflict_unload contact csh_limit domainname editor extended_default extra_siteconfig hide_auto_loaded home icase ignore_cache ignore_cache_subcmds ignore_user_rc ignored_dirs implicit_default implicit_requirement init_envvars info_extension linked_envvars list_json_output list_output list_terse_output locked_configs logged_events logger mcookie_check mcookie_version_check ml modulepath_ignore nearly_forbidden_days non_exportable_tags pager paginate path_entry_reorder protected_envvars quarantine_support rcfile redirect_output require_via reset_target_state run_quarantine search_match set_shell_startup shells_with_ksh_fpath silent_shell_debug siteconfig source_cache spider_indepth spider_json_output spider_output spider_terse_output sticky_purge tag_abbrev tag_color_name tcl_ext_lib tcl_linter term_background term_width unique_name_loaded unload_match_order variant_shortcut verbosity wa_277" complete -f -n '__fish_module_no_subcommand' -c module -a 'help' --description 'Print this or modulefile(s) help info' complete -f -n '__fish_module_no_subcommand' -c module -a 'avail' --description 'List all or matching available modules' diff --git a/init/zsh-functions/_module.in b/init/zsh-functions/_module.in index 7ddbe4c09..29925d1e2 100644 --- a/init/zsh-functions/_module.in +++ b/init/zsh-functions/_module.in @@ -409,7 +409,7 @@ _module() { _arguments \ '--dump-state[Report each state value of current Modules execution]' \ '--reset[Unset environment variable relative to configuration key]' \ - '1:configuration key:(abort_on_error advanced_version_spec auto_handling avail_indepth avail_output avail_terse_output cache_buffer_bytes cache_expiry_secs collection_pin_version collection_pin_tag collection_target color colors conflict_unload contact csh_limit domainname editor extended_default extra_siteconfig hide_auto_loaded home icase ignore_cache ignore_cache_subcmds ignore_user_rc ignored_dirs implicit_default implicit_requirement init_envvars info_extension linked_envvars list_output list_terse_output locked_configs logged_events logger mcookie_check mcookie_version_check ml modulepath_ignore nearly_forbidden_days non_exportable_tags pager paginate path_entry_reorder protected_envvars quarantine_support rcfile redirect_output require_via reset_target_state run_quarantine search_match set_shell_startup shells_with_ksh_fpath silent_shell_debug siteconfig source_cache spider_indepth spider_output spider_terse_output sticky_purge tag_abbrev tag_color_name tcl_ext_lib tcl_linter term_background term_width unique_name_loaded unload_match_order variant_shortcut verbosity wa_277)' \ + '1:configuration key:(abort_on_error advanced_version_spec auto_handling avail_indepth avail_json_output avail_output avail_terse_output cache_buffer_bytes cache_expiry_secs collection_pin_version collection_pin_tag collection_target color colors conflict_unload contact csh_limit domainname editor extended_default extra_siteconfig hide_auto_loaded home icase ignore_cache ignore_cache_subcmds ignore_user_rc ignored_dirs implicit_default implicit_requirement init_envvars info_extension linked_envvars list_json_output list_output list_terse_output locked_configs logged_events logger mcookie_check mcookie_version_check ml modulepath_ignore nearly_forbidden_days non_exportable_tags pager paginate path_entry_reorder protected_envvars quarantine_support rcfile redirect_output require_via reset_target_state run_quarantine search_match set_shell_startup shells_with_ksh_fpath silent_shell_debug siteconfig source_cache spider_indepth spider_json_output spider_output spider_terse_output sticky_purge tag_abbrev tag_color_name tcl_ext_lib tcl_linter term_background term_width unique_name_loaded unload_match_order variant_shortcut verbosity wa_277)' \ && ret=0 ;; (edit) diff --git a/site.exp.in b/site.exp.in index 8ea0fb989..c9a4ec2df 100644 --- a/site.exp.in +++ b/site.exp.in @@ -122,6 +122,9 @@ set install_listoutput "@listoutput@" set install_listterseoutput "@listterseoutput@" set install_spideroutput "@spideroutput@" set install_spiderterseoutput "@spiderterseoutput@" +set install_availjsonoutput "@availjsonoutput@" +set install_listjsonoutput "@listjsonoutput@" +set install_spiderjsonoutput "@spiderjsonoutput@" set install_variantshortcut "@variantshortcut@" diff --git a/tcl/init.tcl.in b/tcl/init.tcl.in index b70f0459e..142051506 100644 --- a/tcl/init.tcl.in +++ b/tcl/init.tcl.in @@ -79,6 +79,9 @@ array set g_config_defs [list\ eltlist}\ auto_handling {MODULES_AUTO_HANDLING @autohandling@ 0 b {0 1}}\ avail_indepth {MODULES_AVAIL_INDEPTH @availindepth@ 0 b {0 1}}\ + avail_json_output {MODULES_AVAIL_JSON_OUTPUT {@availjsonoutput@} 0 l\ + {modulepath alias provided-alias dirwsym sym tag hidden variant\ + variantifspec via} {} {} eltlist}\ avail_output {MODULES_AVAIL_OUTPUT {@availoutput@} 0 l {modulepath alias\ provided-alias dirwsym indesym sym tag key hidden variant variantifspec\ via} {} {} eltlist}\ @@ -114,6 +117,8 @@ array set g_config_defs [list\ initConfInitEnvvars}\ linked_envvars {MODULES_LINKED_ENVVARS {} 0 l {} {}\ initConfLinkedEnvvars}\ + list_json_output {MODULES_LIST_JSON_OUTPUT {@listjsonoutput@} 0 l\ + {variant alias sym tag hidden via} {} {} eltlist}\ list_output {MODULES_LIST_OUTPUT {@listoutput@} 0 l {header idx variant\ alias indesym sym tag hidden key} {} {} eltlist}\ list_terse_output {MODULES_LIST_TERSE_OUTPUT {@listterseoutput@} 0 l\ @@ -147,6 +152,9 @@ array set g_config_defs [list\ 1}}\ siteconfig {{} {@etcdir@/siteconfig.tcl} 0 s}\ spider_indepth {MODULES_SPIDER_INDEPTH @spiderindepth@ 0 b {0 1}}\ + spider_json_output {MODULES_SPIDER_JSON_OUTPUT {@spiderjsonoutput@} 0 l\ + {modulepath alias provided-alias dirwsym sym tag hidden variant\ + variantifspec via} {} {} eltlist}\ spider_output {MODULES_SPIDER_OUTPUT {@spideroutput@} 0 l {modulepath\ alias provided-alias dirwsym indesym sym tag key hidden variant\ variantifspec via} {} {} eltlist}\ diff --git a/tcl/main.tcl.in b/tcl/main.tcl.in index 4a90e0634..57b6de329 100644 --- a/tcl/main.tcl.in +++ b/tcl/main.tcl.in @@ -300,7 +300,7 @@ proc parseModuleCommandArgs {topcall cmd ignerr ml_spec args} { } if {[info exists asked_output]} { - if {[getState report_format] in [list long json]} { + if {[isStateEqual report_format long]} { knerror "Unsupported option '$output_arg' on [getState\ report_format] output mode" } else { diff --git a/tcl/modfind.tcl.in b/tcl/modfind.tcl.in index 1b034374e..c564851d9 100644 --- a/tcl/modfind.tcl.in +++ b/tcl/modfind.tcl.in @@ -980,6 +980,9 @@ proc getVariantList {mod {report 0} {excl_dflval 0} {from 0}} { } elseif {$report == 3} { set vrarr($vrname) [list $vrvalue [expr {$vrvalisdfl > 0 &&\ $vrvalisdfl < 3}]] + # all possible values are reported on json avail output + } elseif {$report == 4 && $from == 2} { + set vrarr($vrname) $vrvalues } else { set vrarr($vrname) $vrvalue } @@ -989,7 +992,12 @@ proc getVariantList {mod {report 0} {excl_dflval 0} {from 0}} { set ret {} foreach vrname [lsort -dictionary [array names vrarr]] { switch -- $report { - 4 {lappend ret $vrname $vrarr($vrname) $vrisbool($vrname)} + 4 { + # value is an array of possible values on avail, a boolean or a + # string on list + lappend ret $vrname $vrarr($vrname) [expr {$from == 2 ? {a} :\ + ($vrisbool($vrname) ? {b} : {s})}] + } 3 {lappend ret $vrname [lindex $vrarr($vrname) 0] [lindex\ $vrarr($vrname) 1]} 2 {lappend ret [list $vrname 0 $vrisbool($vrname) $vrarr($vrname)]} diff --git a/tcl/report.tcl.in b/tcl/report.tcl.in index a2091ba45..08d73e7df 100644 --- a/tcl/report.tcl.in +++ b/tcl/report.tcl.in @@ -1360,16 +1360,18 @@ proc formatArrayValToJson {vallist} { {[]}}] } +# format a list of key/value/type triplets as a json object (value type is +# 'a' for array, 'b' for boolean or 's' for string) proc formatObjectValToJson {objlist} { - foreach {key val isbool} $objlist { + foreach {key val vtype} $objlist { if {[info exists disp]} { append disp {, } } append disp "\"$key\": " - if {$isbool} { - append disp [expr {$val ? {true} : {false}}] - } else { - append disp "\"$val\"" + switch -- $vtype { + a {append disp [formatArrayValToJson $val]} + b {append disp [expr {$val ? {true} : {false}}]} + s {append disp "\"$val\""} } } ##nagelfar ignore Bad expression @@ -1477,13 +1479,12 @@ proc reportModules {search_queries header hsgrkey hstyle show_mtime show_idx\ set report_alias [expr {[isEltInReport alias] || [isEltInReport\ provided-alias]}] # enable variant report if variantifspec configured and some variant is - # specified in query or variant configured for report or list sub-command - # json output + # specified in query or variant configured for report set report_variant [expr {($variant_spec_in_query && [isEltInReport\ - variantifspec 0]) || [isEltInReport variant [expr {[currentState\ - commandname] eq {list} && $json}]]}] + variantifspec 0]) || [isEltInReport variant]}] set collect_variant_from [expr {[currentState commandname] in {avail\ spider} ? {2} : {0}}] + set report_via [isEltInReport via] # prepare list of tag abbreviations that can be substituted and list of # tags whose name should be colored @@ -1498,8 +1499,8 @@ proc reportModules {search_queries header hsgrkey hstyle show_mtime show_idx\ himatcharrvrmap himatcharrvrvalmap set clean_list {} set vr_list {} - set via_list [expr {[isEltInReport via] ? [getViaListModuleForModulepath\ - $header] : {}}] + set via_list [expr {$report_via ? [getViaListModuleForModulepath $header]\ + : {}}] # treat elements in specified order if any ##nagelfar ignore #2 Badly formed if statement @@ -1525,7 +1526,8 @@ proc reportModules {search_queries header hsgrkey hstyle show_mtime show_idx\ if {$json} { ##nagelfar ignore +2 Found constant set dispsgr [formatListEltToJsonDisplay $elt type s directory\ - 1 symbols a $sym_list 1 via a $via_list 1] + 1 symbols a $sym_list $report_sym via a $via_list\ + $report_via] } elseif {$show_mtime} { # append / char after name to clearly indicate this is a dir lassign [formatListEltToLongDisplay $elt di / $sym_list sy {}\ @@ -1540,9 +1542,9 @@ proc reportModules {search_queries header hsgrkey hstyle show_mtime show_idx\ if {$json} { ##nagelfar ignore +4 Found constant set dispsgr [formatListEltToJsonDisplay $elt type s modulefile\ - 1 variants o $vr_list $report_variant symbols a $sym_list 1\ - tags a $tag_list 1 pathname s [lindex $mod_list($elt) 2] 1\ - via a $via_list 1] + 1 variants o $vr_list $report_variant symbols a $sym_list\ + $report_sym tags a $tag_list $report_tag pathname s [lindex\ + $mod_list($elt) 2] 1 via a $via_list $report_via] } elseif {$show_mtime} { set clock_mtime [expr {[lindex $mod_list($elt) 1] ne {} ?\ [clock format [lindex $mod_list($elt) 1] -format {%Y/%m/%d\ @@ -1560,11 +1562,12 @@ proc reportModules {search_queries header hsgrkey hstyle show_mtime show_idx\ } } alias { - if {$json} { + if {$json && $report_alias} { ##nagelfar ignore +3 Found constant set dispsgr [formatListEltToJsonDisplay $elt type s alias 1\ - symbols a $sym_list 1 tags a $tag_list 1 target s [lindex\ - $mod_list($elt) 1] 1 via a $via_list 1] + symbols a $sym_list $report_sym tags a $tag_list\ + $report_tag target s [lindex $mod_list($elt) 1] 1 via a\ + $via_list $report_via] } elseif {$show_mtime} { lassign [formatListEltToLongDisplay $elt al " -> [lindex\ $mod_list($elt) 1]" $sym_list sy {} $default_colored\ diff --git a/testsuite/install.00-init/010-environ.exp b/testsuite/install.00-init/010-environ.exp index 81b4aa940..b43bf9559 100644 --- a/testsuite/install.00-init/010-environ.exp +++ b/testsuite/install.00-init/010-environ.exp @@ -151,6 +151,9 @@ setenv_var MODULES_LIST_OUTPUT {header:idx:variant:sym:tag:key} setenv_var MODULES_LIST_TERSE_OUTPUT {header} setenv_var MODULES_SPIDER_OUTPUT {modulepath:alias:dirwsym:sym:tag:variantifspec:via:key} setenv_var MODULES_SPIDER_TERSE_OUTPUT {modulepath:alias:dirwsym:sym:tag:variantifspec} +setenv_var MODULES_AVAIL_JSON_OUTPUT {modulepath:alias:dirwsym:sym:tag:via} +setenv_var MODULES_LIST_JSON_OUTPUT {variant:sym:tag:via} +setenv_var MODULES_SPIDER_JSON_OUTPUT {modulepath:alias:dirwsym:sym:tag:via} # clean any variant config setenv_var MODULES_VARIANT_SHORTCUT {} diff --git a/testsuite/modules.00-init/010-environ.exp b/testsuite/modules.00-init/010-environ.exp index a03620490..ebbb879fc 100644 --- a/testsuite/modules.00-init/010-environ.exp +++ b/testsuite/modules.00-init/010-environ.exp @@ -152,6 +152,9 @@ setenv_var MODULES_LIST_OUTPUT {header:idx:variant:sym:tag:key} setenv_var MODULES_LIST_TERSE_OUTPUT {header} setenv_var MODULES_SPIDER_OUTPUT {modulepath:alias:dirwsym:sym:tag:variantifspec:via:key} setenv_var MODULES_SPIDER_TERSE_OUTPUT {modulepath:alias:dirwsym:sym:tag:variantifspec} +setenv_var MODULES_AVAIL_JSON_OUTPUT {modulepath:alias:dirwsym:sym:tag:via} +setenv_var MODULES_LIST_JSON_OUTPUT {variant:sym:tag:via} +setenv_var MODULES_SPIDER_JSON_OUTPUT {modulepath:alias:dirwsym:sym:tag:via} # clean any variant config setenv_var MODULES_VARIANT_SHORTCUT {} diff --git a/testsuite/modules.51-scan/020-avail_output-variant.exp b/testsuite/modules.51-scan/020-avail_output-variant.exp index f8cbe01ac..440beac57 100644 --- a/testsuite/modules.51-scan/020-avail_output-variant.exp +++ b/testsuite/modules.51-scan/020-avail_output-variant.exp @@ -392,10 +392,19 @@ set tserr "{\"$mp\": { }}" testouterr_cmd sh {avail -j variant} OK $tserr -set tserr "{\"errors\": \[ -{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \] -}" -testouterr_cmd sh {avail -j -o variant variant} ERR $tserr +set tserr "{\"variant/1.0\": { \"name\": \"variant/1.0\", \"type\": \"modulefile\", \"variants\": { \"bar\": \[ \"val1\", \"val2\", \"val3\" \], \"foo\": \[ \"val1\" \] }, \"pathname\": \"$mp/variant/1.0\"}, +\"variant/2.0\": { \"name\": \"variant/2.0\", \"type\": \"modulefile\", \"variants\": { \"foo\": \[ \"val1\" \] }, \"pathname\": \"$mp/variant/2.0\"}, +\"variant/2.1/2/1\": { \"name\": \"variant/2.1/2/1\", \"type\": \"modulefile\", \"variants\": {}, \"pathname\": \"$mp/variant/2.1/2/1\"}, +\"variant/3.0\": { \"name\": \"variant/3.0\", \"type\": \"modulefile\", \"variants\": { \"bar\": \[ \"val1\", \"val2\" \] }, \"pathname\": \"$mp/variant/3.0\"}, +\"variant/4.0\": { \"name\": \"variant/4.0\", \"type\": \"modulefile\", \"variants\": {}, \"pathname\": \"$mp/variant/4.0\"}, +\"variant/5.0\": { \"name\": \"variant/5.0\", \"type\": \"modulefile\", \"variants\": {}, \"pathname\": \"$mp/variant/5.0\"}, +\"variant/5.1\": { \"name\": \"variant/5.1\", \"type\": \"modulefile\", \"variants\": { \"foo\": \[ \"val1\", \"val2\", \"val3\" \] }, \"pathname\": \"$mp/variant/5.1\"}, +\"variant/6.0\": { \"name\": \"variant/6.0\", \"type\": \"modulefile\", \"variants\": {}, \"pathname\": \"$mp/variant/6.0\"}, +\"variant/7.0\": { \"name\": \"variant/7.0\", \"type\": \"modulefile\", \"variants\": { \"foo\": \[ \"val1\", \"val2\", \"val3\" \] }, \"pathname\": \"$mp/variant/7.0\"}, +\"variant/7.1\": { \"name\": \"variant/7.1\", \"type\": \"modulefile\", \"variants\": {}, \"pathname\": \"$mp/variant/7.1\"}, +\"variant/8.0\": { \"name\": \"variant/8.0\", \"type\": \"modulefile\", \"variants\": {}, \"pathname\": \"$mp/variant/8.0\"}, +\"variant/w s\": { \"name\": \"variant/w s\", \"type\": \"modulefile\", \"variants\": {}, \"pathname\": \"$mp/variant/w s\"}}" +testouterr_cmd sh {avail -j -o variant variant} OK $tserr unsetenv_var TESTSUITE_VARIANT diff --git a/testsuite/modules.70-maint/220-config.exp b/testsuite/modules.70-maint/220-config.exp index 6e3addcc1..3028e3ace 100644 --- a/testsuite/modules.70-maint/220-config.exp +++ b/testsuite/modules.70-maint/220-config.exp @@ -43,6 +43,9 @@ unsetenv_var MODULES_LIST_OUTPUT unsetenv_var MODULES_LIST_TERSE_OUTPUT unsetenv_var MODULES_SPIDER_OUTPUT unsetenv_var MODULES_SPIDER_TERSE_OUTPUT +unsetenv_var MODULES_AVAIL_JSON_OUTPUT +unsetenv_var MODULES_LIST_JSON_OUTPUT +unsetenv_var MODULES_SPIDER_JSON_OUTPUT unsetenv_var MODULES_VARIANT_SHORTCUT if {[info exists env(MODULES_ICASE)]} { set saved_icase $env(MODULES_ICASE) @@ -62,6 +65,7 @@ array set configdfl [list\ advanced_version_spec [expr {$install_advversspec eq {y}}]\ auto_handling [expr {$install_autohandling eq {y}}]\ avail_indepth [expr {$install_availindepth eq {y}}]\ + avail_json_output $install_availjsonoutput\ avail_output $install_availoutput\ avail_terse_output $install_availterseoutput\ csh_limit 4000\ @@ -90,6 +94,7 @@ array set configdfl [list\ init_envvars $install_initenvvars\ info_extension 0\ linked_envvars {}\ + list_json_output $install_listjsonoutput\ list_output $install_listoutput\ list_terse_output $install_listterseoutput\ locked_configs $install_lockedconfigs\ @@ -118,6 +123,7 @@ array set configdfl [list\ siteconfig $siteconfig_filere\ source_cache [expr {$install_sourcecache eq {y}}]\ spider_indepth [expr {$install_spiderindepth eq {y}}]\ + spider_json_output $install_spiderjsonoutput\ spider_output $install_spideroutput\ spider_terse_output $install_spiderterseoutput\ sticky_purge $install_stickypurge\ @@ -139,6 +145,7 @@ array set configvar [list\ advanced_version_spec MODULES_ADVANCED_VERSION_SPEC\ auto_handling MODULES_AUTO_HANDLING\ avail_indepth MODULES_AVAIL_INDEPTH\ + avail_json_output MODULES_AVAIL_JSON_OUTPUT\ avail_output MODULES_AVAIL_OUTPUT\ avail_terse_output MODULES_AVAIL_TERSE_OUTPUT\ cache_buffer_bytes MODULES_CACHE_BUFFER_BYTES\ @@ -165,6 +172,7 @@ array set configvar [list\ init_envvars MODULES_INIT_ENVVARS\ info_extension MODULES_INFO_EXTENSION\ linked_envvars MODULES_LINKED_ENVVARS\ + list_json_output MODULES_LIST_JSON_OUTPUT\ list_output MODULES_LIST_OUTPUT\ list_terse_output MODULES_LIST_TERSE_OUTPUT\ logged_events MODULES_LOGGED_EVENTS\ @@ -191,6 +199,7 @@ array set configvar [list\ silent_shell_debug MODULES_SILENT_SHELL_DEBUG\ source_cache MODULES_SOURCE_CACHE\ spider_indepth MODULES_SPIDER_INDEPTH\ + spider_json_output MODULES_SPIDER_JSON_OUTPUT\ spider_output MODULES_SPIDER_OUTPUT\ spider_terse_output MODULES_SPIDER_TERSE_OUTPUT\ sticky_purge MODULES_STICKY_PURGE\ @@ -211,6 +220,7 @@ array set configvalid [list\ advanced_version_spec {0 1}\ auto_handling {0 1}\ avail_indepth {0 1}\ + avail_json_output {modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via}\ avail_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec via}\ avail_terse_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec}\ cache_buffer_bytes {4096 1000000}\ @@ -227,6 +237,7 @@ array set configvalid [list\ implicit_default {0 1}\ implicit_requirement {0 1}\ info_extension {0 1}\ + list_json_output {variant alias sym tag hidden via}\ list_output {header idx variant alias indesym sym tag hidden key}\ list_terse_output {header idx variant alias indesym sym tag hidden key}\ logged_events {auto_eval requested_eval requested_cmd}\ @@ -246,6 +257,7 @@ array set configvalid [list\ silent_shell_debug {0 1}\ source_cache {0 1}\ spider_indepth {0 1}\ + spider_json_output {modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via}\ spider_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec via}\ spider_terse_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec}\ sticky_purge {error warning silent}\ @@ -259,15 +271,18 @@ array set configvalid [list\ array set configvalidkind [list\ abort_on_error eltlist\ + avail_json_output eltlist\ avail_output eltlist\ avail_terse_output eltlist\ cache_buffer_bytes intbe\ cache_expiry_secs intbe\ + list_json_output eltlist\ list_output eltlist\ list_terse_output eltlist\ logged_events eltlist\ nearly_forbidden_days intbe\ shells_with_ksh_fpath eltlist\ + spider_json_output eltlist\ spider_output eltlist\ spider_terse_output eltlist\ term_width intbe\ @@ -282,11 +297,13 @@ array set configlockable [list\ array set configkind [list\ abort_on_error l\ + avail_json_output l\ avail_output l\ avail_terse_output l\ colors l\ init_envvars l\ linked_envvars l\ + list_json_output l\ list_output l\ list_terse_output l\ logged_events l\ @@ -294,6 +311,7 @@ array set configkind [list\ protected_envvars l\ rcfile l\ shells_with_ksh_fpath l\ + spider_json_output l\ spider_output l\ spider_terse_output l\ tag_abbrev l\ diff --git a/testsuite/modules.70-maint/360-avail_output.exp b/testsuite/modules.70-maint/360-avail_output.exp index 4603f2ffb..a0c7d9db5 100644 --- a/testsuite/modules.70-maint/360-avail_output.exp +++ b/testsuite/modules.70-maint/360-avail_output.exp @@ -58,12 +58,13 @@ testouterr_cmd sh {--output avail foo} ERR "$error_msgs: Missing value for '--ou # mix with --terse/--long/--json testouterr_cmd_re sh {avail -o sym --terse foo/1.0} OK $tserr_avail_terse_foo testouterr_cmd_re sh {avail -t --output=sym foo/1.0} OK $tserr_avail_terse_foo -testouterr_cmd sh {avail -j --output=sym foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '--output' on json output mode\" \] } \]\n}" +set tserr_avail_json_foo "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"symbols\": \[ \"bar\" \], \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {avail -j --output=sym foo/1.0} OK $tserr_avail_json_foo testouterr_cmd sh {avail -o sym -l foo/1.0} ERR "$error_msgs: Unsupported option '-o' on long output mode" testouterr_cmd sh {avail --long --output=sym foo/1.0} ERR "$error_msgs: Unsupported option '--output' on long output mode" -testouterr_cmd sh {avail -j --output=sym foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '--output' on json output mode\" \] } \]\n}" -testouterr_cmd sh {avail -o sym --json foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \]\n}" -testouterr_cmd sh {avail -o sym --long -j foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \]\n}" +testouterr_cmd sh {avail -o sym --json foo/1.0} OK $tserr_avail_json_foo +testouterr_cmd sh {avail -o sym --long -j foo/1.0} OK $tserr_avail_json_foo +testouterr_cmd sh {avail -o sym -j --long foo/1.0} ERR "$error_msgs: Unsupported option '-o' on long output mode" # test with ml shortcut command testouterr_cmd_re sh {ml av -o sym foo/1.0} OK $tserr_avail_foo @@ -71,7 +72,7 @@ testouterr_cmd sh {ml av -o} ERR "$error_msgs: Missing value for '-o' option\n$e testouterr_cmd_re sh {ml --output=sym av foo/1.0} OK $tserr_avail_foo testouterr_cmd sh {--output ml av foo} ERR "$error_msgs: Missing value for '--output' option\n$err_typehelp" testouterr_cmd sh {ml av --long --output=sym foo/1.0} ERR "$error_msgs: Unsupported option '--output' on long output mode" -testouterr_cmd sh {--json ml av -o sym foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \]\n}" +testouterr_cmd sh {--json ml av -o sym foo/1.0} OK $tserr_avail_json_foo # test on a sub-command that does not support these options set ans [list] @@ -106,6 +107,19 @@ testouterr_cmd sh {avail -o header foo} ERR $tserr_invalid_o_val testouterr_cmd sh {avail --output=idx foo} ERR $tserr_invalid_output_val testouterr_cmd sh {avail --output=header:key foo} ERR $tserr_invalid_output_val +# invalid value set on json output mode +set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on avail sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on avail sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" +testouterr_cmd sh {avail -j --output=foo foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {avail -j --output=sym,alias foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {avail -j -o foo foo} ERR $tserr_invalid_json_o_val +testouterr_cmd sh {ml av -j -o +foo foo} ERR $tserr_invalid_json_o_val +# element accepted on regular/terse output mode but not on json +testouterr_cmd sh {avail -j --output=key foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {avail -j --output=sym:indesym foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {avail -j -o header foo} ERR $tserr_invalid_json_o_val +testouterr_cmd sh {avail -j -o idx foo} ERR $tserr_invalid_json_o_val + # test usage in modulefile set ans [list] lappend ans [list set _LMFILES_ $mp/output/1.0] @@ -1000,6 +1014,108 @@ testouterr_cmd sh {avail --output=+variant --terse foo/0.2} OK {foo/0.2(sym) \n\nKey:\n\\(symbolic-version\\) " +testouterr_cmd_re sh {avail -t foo/1.0} OK "$mpre:\nfoo/1.0\\(bar\\) " +# -o/--output supersedes environment variable +testouterr_cmd sh {avail -j -o modulepath foo/1.0} OK "{\"$mp\": {\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\"}\n}}" +testouterr_cmd sh {avail -j --output=+modulepath foo/1.0} OK "{\"$mp\": {\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"symbols\": \[ \"bar\" \], \"tags\": \[ \"baz\" \], \"pathname\": \"$mp/foo/1.0\"}\n}}" +testouterr_cmd sh {avail -j --output=-tag foo/1.0} OK "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"symbols\": \[ \"bar\" \], \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {ml av -j -o -tag:sym foo/1.0} OK $tserr_avail_json_min +# invalid value set (ignored) +setenv_var MODULES_AVAIL_JSON_OUTPUT foo +testouterr_cmd sh {avail -j foo/1.0} OK $tserr_avail_json +setenv_var MODULES_AVAIL_JSON_OUTPUT sym,tag +testouterr_cmd sh {avail -j foo/1.0} OK $tserr_avail_json +setenv_var MODULES_AVAIL_JSON_OUTPUT key:sym +testouterr_cmd sh {avail -j foo/1.0} OK $tserr_avail_json +unsetenv_var MODULES_AVAIL_JSON_OUTPUT + +# variant and variantifspec elements +setenv_var TESTSUITE_VARIANT 13 +set tserr_avail_json_variant "{\"variant/1.0\": { \"name\": \"variant/1.0\", \"type\": \"modulefile\", \"variants\": { \"bar\": \[ \"val1\", \"val2\", \"val3\" \], \"foo\": \[ \"val1\" \] }, \"pathname\": \"$mp/variant/1.0\"}}" +testouterr_cmd sh {avail -j -o variant variant/1.0} OK $tserr_avail_json_variant +testouterr_cmd sh {avail -j -o variantifspec variant/1.0 foo=val1} OK $tserr_avail_json_variant +testouterr_cmd sh {avail -j -o variantifspec variant/1.0} OK "{\"variant/1.0\": { \"name\": \"variant/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/variant/1.0\"}}" +# possible values of boolean variant +testouterr_cmd sh {avail -j -o variant refresh/2.0} OK "{\"refresh/2.0\": { \"name\": \"refresh/2.0\", \"type\": \"modulefile\", \"variants\": { \"foo\": \[ \"on\", \"off\" \] }, \"pathname\": \"$mp/refresh/2.0\"}}" +unsetenv_var TESTSUITE_VARIANT + +# hidden element +set mp $modpath.2 +set mpre [regsub -all "\(\[.+?\]\)" $mp {\\\1}] +setenv_path_var MODULEPATH $mp +setenv_var TESTSUITE_HIDE2_SET1 1 +set tserr_hide2_json_10 "\"hide2/1.0\": { \"name\": \"hide2/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/hide2/1.0\"}" +set tserr_hide2_json_15 "\"hide2/1.5\": { \"name\": \"hide2/1.5\", \"type\": \"modulefile\", \"pathname\": \"$mp/hide2/1.0\"}" +testouterr_cmd sh {avail -j -o modulepath hide2} OK "{\"$mp\": {\n$tserr_hide2_json_15\n}}" +testouterr_cmd sh {avail -j -o modulepath:hidden hide2} OK "{\"$mp\": {\n$tserr_hide2_json_10,\n$tserr_hide2_json_15\n}}" +testouterr_cmd sh {avail -j --all -o modulepath hide2} OK "{\"$mp\": {\n$tserr_hide2_json_10,\n$tserr_hide2_json_15\n}}" +unsetenv_var TESTSUITE_HIDE2_SET1 + +# alias element +testouterr_cmd sh {avail -j -o modulepath hide2} OK "{\"$mp\": {\n$tserr_hide2_json_10,\n$tserr_hide2_json_15\n}}" +testouterr_cmd sh {avail -j -o modulepath:alias hide2} OK "{\"$mp\": {\n$tserr_hide2_json_10,\n\"hide2/1.1\": { \"name\": \"hide2/1.1\", \"type\": \"alias\", \"target\": \"hide2/1.0\"},\n$tserr_hide2_json_15\n}}" +testouterr_cmd sh {avail -j -o modulepath:alias:sym:tag hide2} OK "{\"$mp\": { +\"hide2/1.0\": { \"name\": \"hide2/1.0\", \"type\": \"modulefile\", \"symbols\": \[ \"1.2\" \], \"tags\": \[\], \"pathname\": \"$mp/hide2/1.0\"}, +\"hide2/1.1\": { \"name\": \"hide2/1.1\", \"type\": \"alias\", \"symbols\": \[\], \"tags\": \[\], \"target\": \"hide2/1.0\"}, +\"hide2/1.5\": { \"name\": \"hide2/1.5\", \"type\": \"modulefile\", \"symbols\": \[\], \"tags\": \[\], \"pathname\": \"$mp/hide2/1.0\"} +}}" + +# dirwsym element +testouterr_cmd sh {avail -j -o modulepath:sym hide3/sub2} OK "{\"$mp\": { +\"hide3/sub2/4.0\": { \"name\": \"hide3/sub2/4.0\", \"type\": \"modulefile\", \"symbols\": \[ \"default\", \"sym\" \], \"pathname\": \"$mp/hide3/sub2/4.0\"}, +\"hide3/sub2/5.0\": { \"name\": \"hide3/sub2/5.0\", \"type\": \"modulefile\", \"symbols\": \[ \"new\" \], \"pathname\": \"$mp/hide3/sub2/5.0\"} +}}" +testouterr_cmd sh {avail -j -o modulepath:dirwsym:sym hide3/sub2} OK "{\"$mp\": { +\"hide3/sub2\": { \"name\": \"hide3/sub2\", \"type\": \"directory\", \"symbols\": \[ \"sym\" \]}, +\"hide3/sub2/4.0\": { \"name\": \"hide3/sub2/4.0\", \"type\": \"modulefile\", \"symbols\": \[ \"default\", \"sym\" \], \"pathname\": \"$mp/hide3/sub2/4.0\"}, +\"hide3/sub2/5.0\": { \"name\": \"hide3/sub2/5.0\", \"type\": \"modulefile\", \"symbols\": \[ \"new\" \], \"pathname\": \"$mp/hide3/sub2/5.0\"} +}}" +testouterr_cmd sh {avail -j -o modulepath:dirwsym hide3/sub2} OK "{\"$mp\": { +\"hide3/sub2\": { \"name\": \"hide3/sub2\", \"type\": \"directory\"}, +\"hide3/sub2/4.0\": { \"name\": \"hide3/sub2/4.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/hide3/sub2/4.0\"}, +\"hide3/sub2/5.0\": { \"name\": \"hide3/sub2/5.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/hide3/sub2/5.0\"} +}}" + +# provided-alias element +set mp $modpath.4 +set mpre [regsub -all "\(\[.+?\]\)" $mp {\\\1}] +setenv_path_var MODULEPATH $mp +setenv_var TESTSUITE_PROVIDE multiple +testouterr_cmd sh {avail -j -o modulepath:alias foo/1} OK "{\"$mp\": {\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\"}\n}}" +testouterr_cmd sh {avail -j -o modulepath:provided-alias foo/1} OK "{\"$mp\": {\n\"foo/1\": { \"name\": \"foo/1\", \"type\": \"alias\", \"target\": \"provide/1\"},\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\"}\n}}" +unsetenv_var TESTSUITE_PROVIDE + + # # Cleanup # diff --git a/testsuite/modules.70-maint/361-list_output.exp b/testsuite/modules.70-maint/361-list_output.exp index 949795551..3817533f7 100644 --- a/testsuite/modules.70-maint/361-list_output.exp +++ b/testsuite/modules.70-maint/361-list_output.exp @@ -57,7 +57,10 @@ testouterr_cmd sh {--output list} ERR "$error_msgs: Missing value for '--output' # mix with --terse/--long/--json testouterr_cmd sh {list -o sym --terse} OK $tserr_list_terse_foo testouterr_cmd sh {list --long --output=sym} ERR "$error_msgs: Unsupported option '--output' on long output mode" -testouterr_cmd sh {list -j --output=sym} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '--output' on json output mode\" \] } \]\n}" +set tserr_list_json_foo "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"symbols\": \[ \"bar\" \], \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {list -j --output=sym} OK $tserr_list_json_foo +testouterr_cmd sh {list -o sym --long -j} OK $tserr_list_json_foo +testouterr_cmd sh {list -o sym -j --long} ERR "$error_msgs: Unsupported option '-o' on long output mode" # test with ml shortcut command testouterr_cmd sh {ml -o sym} OK $tserr_list_foo @@ -65,7 +68,7 @@ testouterr_cmd sh {ml -o} ERR "$error_msgs: Missing value for '-o' option\n$err_ testouterr_cmd sh {ml --output=sym} OK $tserr_list_foo testouterr_cmd sh {--output ml} ERR "$error_msgs: Missing value for '--output' option\n$err_typehelp" testouterr_cmd sh {ml -l --output=sym} ERR "$error_msgs: Unsupported option '--output' on long output mode" -testouterr_cmd sh {ml -o sym --json} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \]\n}" +testouterr_cmd sh {ml -o sym --json} OK $tserr_list_json_foo # invalid value set set tserr_invalid_output_val "$error_msgs: Invalid element in value list for '--output' option on list sub-command @@ -78,6 +81,21 @@ testouterr_cmd sh {list --output="sym alias"} ERR $tserr_invalid_output_val testouterr_cmd sh {list -o foo} ERR $tserr_invalid_o_val testouterr_cmd sh {list -o sym,alias} ERR $tserr_invalid_o_val testouterr_cmd sh {list -o "sym alias"} ERR $tserr_invalid_o_val + +# invalid value set on json output mode +set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on list sub-command\", \"Allowed elements are: variant alias sym tag hidden via (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on list sub-command\", \"Allowed elements are: variant alias sym tag hidden via (separated by ':')\" \] } \]\n}" +testouterr_cmd sh {list -j --output=foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {list -j --output=sym,alias} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {list -j -o foo} ERR $tserr_invalid_json_o_val +testouterr_cmd sh {ml -j -o +foo} ERR $tserr_invalid_json_o_val +# element accepted on regular/terse output mode but not on json +testouterr_cmd sh {list -j --output=header} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {list -j --output=sym:idx} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {list -j -o key} ERR $tserr_invalid_json_o_val +testouterr_cmd sh {list -j -o indesym} ERR $tserr_invalid_json_o_val +# element accepted on json output mode only +testouterr_cmd sh {list -t --output=via} ERR $tserr_invalid_output_val testouterr_cmd sh {list -o foo -l} ERR "$error_msgs: Unsupported option '-o' on long output mode" testouterr_cmd sh {ml -o foo} ERR $tserr_invalid_o_val @@ -322,6 +340,58 @@ testouterr_cmd sh {--output=+bar list} ERR $tserr_invalid_output_val testouterr_cmd sh {-t list -o -:sym} ERR $tserr_invalid_o_val +# +# MODULES_LIST_JSON_OUTPUT tests +# + +setenv_loaded_module [list foo/1.0] [list $mp/foo/1.0] +setenv_var __MODULES_LMALTNAME foo/1.0&foo/bar&foo&al|alfoo +setenv_var __MODULES_LMTAG foo/1.0&baz +setenv_var __MODULES_LMVARIANT foo/1.0&bar|1|1|0&baz|val|0|0 +unsetenv_var MODULES_LIST_OUTPUT +unsetenv_var MODULES_LIST_TERSE_OUTPUT + +# default content +set tserr_list_json "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"variants\": { \"bar\": true, \"baz\": \"val\" }, \"symbols\": \[ \"bar\" \], \"tags\": \[ \"baz\" \], \"pathname\": \"$mp/foo/1.0\", \"via\": \[\]}}" +testouterr_cmd sh {list -j} OK $tserr_list_json + +# only intrinsic keys are reported when no element is enabled +set tserr_list_json_min "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {list -j --output=} OK $tserr_list_json_min +testouterr_cmd sh {list -j -o -variant:sym:tag:via} OK $tserr_list_json_min + +setenv_var MODULES_LIST_JSON_OUTPUT tag +testouterr_cmd sh {list -j} OK "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"tags\": \[ \"baz\" \], \"pathname\": \"$mp/foo/1.0\"}}" +# no impact on regular/terse output +testouterr_cmd sh {list -t} OK $cur_loaded\nfoo/1.0 +# -o/--output supersedes environment variable +testouterr_cmd sh {list -j -o sym} OK $tserr_list_json_foo +testouterr_cmd sh {list -j --output=+alias} OK "{\"alfoo\": { \"name\": \"alfoo\", \"type\": \"alias\", \"tags\": \[\], \"target\": \"foo/1.0\"},\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"tags\": \[ \"baz\" \], \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {ml -j --output=-tag} OK $tserr_list_json_min +# invalid value set (ignored) +setenv_var MODULES_LIST_JSON_OUTPUT foo +testouterr_cmd sh {list -j} OK $tserr_list_json +setenv_var MODULES_LIST_JSON_OUTPUT sym,tag +testouterr_cmd sh {list -j} OK $tserr_list_json +setenv_var MODULES_LIST_JSON_OUTPUT header:sym +testouterr_cmd sh {list -j} OK $tserr_list_json +unsetenv_var MODULES_LIST_JSON_OUTPUT + +# alias element +testouterr_cmd sh {list -j -o alias:sym} OK "{\"alfoo\": { \"name\": \"alfoo\", \"type\": \"alias\", \"symbols\": \[\], \"target\": \"foo/1.0\"},\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"symbols\": \[ \"bar\" \], \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {list -j -o alias:sym al} OK "{\"alfoo\": { \"name\": \"alfoo\", \"type\": \"alias\", \"symbols\": \[\], \"target\": \"foo/1.0\"}}" + +# hidden element +setenv_var __MODULES_LMTAG foo/1.0&hidden-loaded +set tserr_list_json_hidden "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"tags\": \[ \"hidden-loaded\" \], \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {list -j -o tag} OK "{}" +testouterr_cmd sh {list -j -o tag:hidden} OK $tserr_list_json_hidden +testouterr_cmd sh {list -j --all -o tag} OK $tserr_list_json_hidden +unsetenv_var __MODULES_LMTAG +unsetenv_var __MODULES_LMVARIANT +unsetenv_var __MODULES_LMALTNAME + + # # Cleanup # diff --git a/testsuite/modules.70-maint/362-spider_output.exp b/testsuite/modules.70-maint/362-spider_output.exp index 3b6b7dcf0..a2aa32b73 100644 --- a/testsuite/modules.70-maint/362-spider_output.exp +++ b/testsuite/modules.70-maint/362-spider_output.exp @@ -58,12 +58,13 @@ testouterr_cmd sh {--output spider foo} ERR "$error_msgs: Missing value for '--o # mix with --terse/--long/--json testouterr_cmd_re sh {spider -o sym --terse foo/1.0} OK $tserr_spider_terse_foo testouterr_cmd_re sh {spider -t --output=sym foo/1.0} OK $tserr_spider_terse_foo -testouterr_cmd sh {spider -j --output=sym foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '--output' on json output mode\" \] } \]\n}" +set tserr_spider_json_foo "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"symbols\": \[ \"bar\" \], \"pathname\": \"$mp/foo/1.0\"}}" +testouterr_cmd sh {spider -j --output=sym foo/1.0} OK $tserr_spider_json_foo testouterr_cmd sh {spider -o sym -l foo/1.0} ERR "$error_msgs: Unsupported option '-o' on long output mode" testouterr_cmd sh {spider --long --output=sym foo/1.0} ERR "$error_msgs: Unsupported option '--output' on long output mode" -testouterr_cmd sh {spider -j --output=sym foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '--output' on json output mode\" \] } \]\n}" -testouterr_cmd sh {spider -o sym --json foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \]\n}" -testouterr_cmd sh {spider -o sym --long -j foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \]\n}" +testouterr_cmd sh {spider -o sym --json foo/1.0} OK $tserr_spider_json_foo +testouterr_cmd sh {spider -o sym --long -j foo/1.0} OK $tserr_spider_json_foo +testouterr_cmd sh {spider -o sym -j --long foo/1.0} ERR "$error_msgs: Unsupported option '-o' on long output mode" # test with ml shortcut command testouterr_cmd_re sh {ml spi -o sym foo/1.0} OK $tserr_spider_foo @@ -71,7 +72,7 @@ testouterr_cmd sh {ml spi -o} ERR "$error_msgs: Missing value for '-o' option\n$ testouterr_cmd_re sh {ml --output=sym spi foo/1.0} OK $tserr_spider_foo testouterr_cmd sh {--output ml spi foo} ERR "$error_msgs: Missing value for '--output' option\n$err_typehelp" testouterr_cmd sh {ml spi --long --output=sym foo/1.0} ERR "$error_msgs: Unsupported option '--output' on long output mode" -testouterr_cmd sh {--json ml spi -o sym foo/1.0} ERR "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Unsupported option '-o' on json output mode\" \] } \]\n}" +testouterr_cmd sh {--json ml spi -o sym foo/1.0} OK $tserr_spider_json_foo # invalid value set set tserr_invalid_output_val "$error_msgs: Invalid element in value list for '--output' option on spider sub-command @@ -92,6 +93,19 @@ testouterr_cmd sh {spider -o header foo} ERR $tserr_invalid_o_val testouterr_cmd sh {spider --output=idx foo} ERR $tserr_invalid_output_val testouterr_cmd sh {spider --output=header:key foo} ERR $tserr_invalid_output_val +# invalid value set on json output mode +set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on spider sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on spider sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" +testouterr_cmd sh {spider -j --output=foo foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {spider -j --output=sym,alias foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {spider -j -o foo foo} ERR $tserr_invalid_json_o_val +testouterr_cmd sh {ml spi -j -o +foo foo} ERR $tserr_invalid_json_o_val +# element accepted on regular/terse output mode but not on json +testouterr_cmd sh {spider -j --output=key foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {spider -j --output=sym:indesym foo} ERR $tserr_invalid_json_output_val +testouterr_cmd sh {spider -j -o header foo} ERR $tserr_invalid_json_o_val +testouterr_cmd sh {spider -j -o idx foo} ERR $tserr_invalid_json_o_val + # # MODULES_SPIDER_OUTPUT/MODULES_SPIDER_TERSE_OUTPUT tests @@ -1004,6 +1018,47 @@ testouterr_cmd sh {spider --output=+variant --terse foo/0.2} OK {foo/0.2(sym) " +# -o/--output supersedes environment variable +testouterr_cmd sh {spider -j -o sym foo/1.0} OK $tserr_spider_json_foo +testouterr_cmd sh {spider -j --output=-via foo/1.0} OK "{\"$mp\": {\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\"}\n}}" +testouterr_cmd sh {ml spi -j --output=+tag foo/1.0} OK "{\"$mp\": {\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"tags\": \[ \"baz\" \], \"pathname\": \"$mp/foo/1.0\", \"via\": \[\]}\n}}" +# invalid value set (ignored) +setenv_var MODULES_SPIDER_JSON_OUTPUT foo +testouterr_cmd sh {spider -j foo/1.0} OK $tserr_spider_json +setenv_var MODULES_SPIDER_JSON_OUTPUT sym,tag +testouterr_cmd sh {spider -j foo/1.0} OK $tserr_spider_json +setenv_var MODULES_SPIDER_JSON_OUTPUT indesym:sym +testouterr_cmd sh {spider -j foo/1.0} OK $tserr_spider_json +unsetenv_var MODULES_SPIDER_JSON_OUTPUT + + # # Cleanup # From 37a34e53720e0d2623b9a725f5f0861ee5690d0d Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Mon, 2 Nov 2026 08:13:13 +0100 Subject: [PATCH 3/4] Add mtime element to JSON output configuration Add the mtime element, specific to JSON output mode, to the avail_json_output, list_json_output and spider_json_output configuration options. When enabled, an mtime key is set on each modulefile JSON object with the modification time of the modulefile as an ISO 8601 string (null if it cannot be fetched). This element is not enabled by default as it requires to stat every modulefile when no module cache file is available. Fixes #615 Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Xavier Delaruelle --- INSTALL.rst | 12 ++++++------ NEWS.rst | 3 +++ configure | 19 ++++++++++--------- doc/source/changes.rst | 3 ++- doc/source/module.rst | 16 ++++++++++++---- tcl/init.tcl.in | 6 +++--- tcl/report.tcl.in | 16 +++++++++++++--- tcl/subcmd.tcl.in | 8 +++++--- .../modules.01-general/120-siteconfig.exp | 4 ++-- testsuite/modules.70-maint/220-config.exp | 6 +++--- .../modules.70-maint/360-avail_output.exp | 19 +++++++++++++++++-- .../modules.70-maint/361-list_output.exp | 19 +++++++++++++++++-- .../modules.70-maint/362-spider_output.exp | 16 ++++++++++++++-- 13 files changed, 107 insertions(+), 40 deletions(-) diff --git a/INSTALL.rst b/INSTALL.rst index 3d55a7ee7..26f307913 100644 --- a/INSTALL.rst +++ b/INSTALL.rst @@ -995,8 +995,8 @@ instance :instopt:`--without-modulepath<--with-modulepath>`): Specify the content to report on avail sub-command JSON output in addition to the available module names. Elements accepted in LIST are: ``modulepath``, ``alias``, ``provided-alias``, ``dirwsym``, ``sym``, - ``tag``, ``hidden``, ``variant``, ``variantifspec`` and ``via`` (elements in - LIST are separated by ``:``). The order of the elements in LIST + ``tag``, ``hidden``, ``variant``, ``variantifspec``, ``via`` and ``mtime`` + (elements in LIST are separated by ``:``). The order of the elements in LIST does not matter. (default=\ ``modulepath:alias:dirwsym:sym:tag:via``) @@ -1279,8 +1279,8 @@ instance :instopt:`--without-modulepath<--with-modulepath>`): Specify the content to report on list sub-command JSON output in addition to the loaded module names. Elements accepted in LIST are: - ``variant``, ``alias``, ``sym``, ``tag``, ``hidden`` and ``via`` (elements - in LIST are separated by ``:``). The order of the elements in LIST + ``variant``, ``alias``, ``sym``, ``tag``, ``hidden``, ``via`` and ``mtime`` + (elements in LIST are separated by ``:``). The order of the elements in LIST does not matter. (default=\ ``variant:sym:tag:via``) @@ -1544,8 +1544,8 @@ instance :instopt:`--without-modulepath<--with-modulepath>`): Specify the content to report on spider sub-command JSON output in addition to the available module names. Elements accepted in LIST are: ``modulepath``, ``alias``, ``provided-alias``, ``dirwsym``, ``sym``, - ``tag``, ``hidden``, ``variant``, ``variantifspec`` and ``via`` (elements in - LIST are separated by ``:``). The order of the elements in LIST + ``tag``, ``hidden``, ``variant``, ``variantifspec``, ``via`` and ``mtime`` + (elements in LIST are separated by ``:``). The order of the elements in LIST does not matter. (default=\ ``modulepath:alias:dirwsym:sym:tag:via``) diff --git a/NEWS.rst b/NEWS.rst index 90df4ecd3..bab14c754 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -48,6 +48,9 @@ Modules 5.8.0 (not yet released) :envvar:`MODULES_LIST_JSON_OUTPUT` and :envvar:`MODULES_SPIDER_JSON_OUTPUT` environment variables or the :option:`--output` command line switch, which is now accepted on :option:`--json` output mode. +* Add *mtime* element to the JSON output configuration of :subcmd:`avail`, + :subcmd:`list` and :subcmd:`spider` sub-commands to report the modification + time of modulefiles. (fix issue #615) .. _5.7 release notes: diff --git a/configure b/configure index 13ae2de98..c7c61504f 100755 --- a/configure +++ b/configure @@ -373,8 +373,8 @@ Optional Packages: JSON output. elements accepted in LIST are: \`modulepath', \`alias', \`provided-alias', \`dirwsym', \`sym', \`tag', \`hidden', \`variant', - \`variantifspec' and \`via' (elements in LIST are - separated by \`:') + \`variantifspec', \`via' and \`mtime' (elements in + LIST are separated by \`:') [$availjsonoutput] --with-avail-output=LIST specify content to report on avail sub-command @@ -438,8 +438,9 @@ Optional Packages: --with-list-json-output=LIST specify content to report on list sub-command JSON output. elements accepted in LIST are: - \`variant', \`alias', \`sym', \`tag', \`hidden' and - \`via' (elements in LIST are separated by \`:') + \`variant', \`alias', \`sym', \`tag', \`hidden', + \`via' and \`mtime' (elements in LIST are separated + by \`:') [$listjsonoutput] --with-list-output=LIST specify content to report on list sub-command @@ -512,8 +513,8 @@ Optional Packages: JSON output. elements accepted in LIST are: \`modulepath', \`alias', \`provided-alias', \`dirwsym', \`sym', \`tag', \`hidden', \`variant', - \`variantifspec' and \`via' (elements in LIST are - separated by \`:') + \`variantifspec', \`via' and \`mtime' (elements in + LIST are separated by \`:') [$spiderjsonoutput] --with-spider-output=LIST specify content to report on spider sub-command @@ -999,13 +1000,13 @@ for arg in "$@"; do allowedval=" modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec " ; check_and_get_package_value_list "spiderterseoutput" "$arg" "$allowedval" "" ;; --with-avail-json-output=*|--without-avail-json-output) - allowedval=" modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via " ; + allowedval=" modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime " ; check_and_get_package_value_list "availjsonoutput" "$arg" "$allowedval" "" ;; --with-list-json-output=*|--without-list-json-output) - allowedval=" variant alias sym tag hidden via " ; + allowedval=" variant alias sym tag hidden via mtime " ; check_and_get_package_value_list "listjsonoutput" "$arg" "$allowedval" "" ;; --with-spider-json-output=*|--without-spider-json-output) - allowedval=" modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via " ; + allowedval=" modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime " ; check_and_get_package_value_list "spiderjsonoutput" "$arg" "$allowedval" "" ;; --with-variant-shortcut=*|--without-variant-shortcut) variantshortcut=$(get_package_value "$arg" "") ;; diff --git a/doc/source/changes.rst b/doc/source/changes.rst index f17fef269..aaa3b27ea 100644 --- a/doc/source/changes.rst +++ b/doc/source/changes.rst @@ -888,7 +888,8 @@ The following command line switches appeared on Modules 5. Starting Modules 5.6, :subcmd:`spider` sub-command is supported. Starting Modules 5.8, this command line switch is accepted on JSON output - mode. + mode and the *mtime* element is added to report the modification time of + modulefiles in JSON output. :option:`--force` diff --git a/doc/source/module.rst b/doc/source/module.rst index bd24c50ab..ea05a6ce8 100644 --- a/doc/source/module.rst +++ b/doc/source/module.rst @@ -548,9 +548,9 @@ switches are accepted: On JSON output mode, accepted elements in *LIST* for :subcmd:`avail` and :subcmd:`spider` sub-command are: *modulepath*, *alias*, *provided-alias*, - *dirwsym*, *sym*, *tag*, *hidden*, *variant*, *variantifspec* and *via*. - Accepted elements for :subcmd:`list` sub-command are: *variant*, *alias*, - *sym*, *tag*, *hidden* and *via*. + *dirwsym*, *sym*, *tag*, *hidden*, *variant*, *variantifspec*, *via* and + *mtime*. Accepted elements for :subcmd:`list` sub-command are: *variant*, + *alias*, *sym*, *tag*, *hidden*, *via* and *mtime*. The order of the elements in *LIST* does not matter. Module names are the only content reported when *LIST* is set to an empty value. @@ -594,7 +594,7 @@ switches are accepted: :subcmd:`spider` sub-commands .. versionchanged:: 5.8 - JSON output mode supported + JSON output mode supported and element *mtime* added for this mode .. option:: --paginate, -p @@ -5077,6 +5077,10 @@ ENVIRONMENT * ``hidden``: show all hidden modules. * ``modulepath``: available modules grouped by modulepath, each modulepath name being a key of the top-level JSON object. + * ``mtime``: ``mtime`` key set on each modulefile JSON object, with + modulefile modification time as value. Time is a string in ISO 8601 format + (``YYYY-MM-DDTHH:MM:SS±hhmm``) or ``null`` if modification time cannot be + fetched. * ``sym``: ``symbols`` key set on each JSON object, with the symbolic versions associated with available module as value. * ``tag``: ``tags`` key set on each JSON object, with the tags associated @@ -5683,6 +5687,10 @@ ENVIRONMENT * ``alias``: module aliases targeting loaded modules. * ``hidden``: show hidden loaded modules. + * ``mtime``: ``mtime`` key set on each modulefile JSON object, with + modulefile modification time as value. Time is a string in ISO 8601 format + (``YYYY-MM-DDTHH:MM:SS±hhmm``) or ``null`` if modification time cannot be + fetched. * ``sym``: ``symbols`` key set on each JSON object, with the symbolic versions associated with loaded module as value. * ``tag``: ``tags`` key set on each JSON object, with the tags associated diff --git a/tcl/init.tcl.in b/tcl/init.tcl.in index 142051506..fbcd3cedb 100644 --- a/tcl/init.tcl.in +++ b/tcl/init.tcl.in @@ -81,7 +81,7 @@ array set g_config_defs [list\ avail_indepth {MODULES_AVAIL_INDEPTH @availindepth@ 0 b {0 1}}\ avail_json_output {MODULES_AVAIL_JSON_OUTPUT {@availjsonoutput@} 0 l\ {modulepath alias provided-alias dirwsym sym tag hidden variant\ - variantifspec via} {} {} eltlist}\ + variantifspec via mtime} {} {} eltlist}\ avail_output {MODULES_AVAIL_OUTPUT {@availoutput@} 0 l {modulepath alias\ provided-alias dirwsym indesym sym tag key hidden variant variantifspec\ via} {} {} eltlist}\ @@ -118,7 +118,7 @@ array set g_config_defs [list\ linked_envvars {MODULES_LINKED_ENVVARS {} 0 l {} {}\ initConfLinkedEnvvars}\ list_json_output {MODULES_LIST_JSON_OUTPUT {@listjsonoutput@} 0 l\ - {variant alias sym tag hidden via} {} {} eltlist}\ + {variant alias sym tag hidden via mtime} {} {} eltlist}\ list_output {MODULES_LIST_OUTPUT {@listoutput@} 0 l {header idx variant\ alias indesym sym tag hidden key} {} {} eltlist}\ list_terse_output {MODULES_LIST_TERSE_OUTPUT {@listterseoutput@} 0 l\ @@ -154,7 +154,7 @@ array set g_config_defs [list\ spider_indepth {MODULES_SPIDER_INDEPTH @spiderindepth@ 0 b {0 1}}\ spider_json_output {MODULES_SPIDER_JSON_OUTPUT {@spiderjsonoutput@} 0 l\ {modulepath alias provided-alias dirwsym sym tag hidden variant\ - variantifspec via} {} {} eltlist}\ + variantifspec via mtime} {} {} eltlist}\ spider_output {MODULES_SPIDER_OUTPUT {@spideroutput@} 0 l {modulepath\ alias provided-alias dirwsym indesym sym tag key hidden variant\ variantifspec via} {} {} eltlist}\ diff --git a/tcl/report.tcl.in b/tcl/report.tcl.in index 08d73e7df..338aae311 100644 --- a/tcl/report.tcl.in +++ b/tcl/report.tcl.in @@ -1378,7 +1378,9 @@ proc formatObjectValToJson {objlist} { return [expr {[info exists disp] ? "{ $disp }" : "{}"}] } -# format an element with its syms for a json display in a list +# format an element with its syms for a json display in a list (value type +# is 'a' for array, 'o' for object, 's' for string or 'r' for raw value +# already formatted in JSON) proc formatListEltToJsonDisplay {elt args} { set disp "\"$elt\": \{ \"name\": \"$elt\"" foreach {key vtype val show} $args { @@ -1390,6 +1392,7 @@ proc formatListEltToJsonDisplay {elt args} { a {append disp [formatArrayValToJson $val]} o {append disp [formatObjectValToJson $val]} s {append disp "\"$val\""} + r {append disp $val} } } append disp "\}" @@ -1485,6 +1488,8 @@ proc reportModules {search_queries header hsgrkey hstyle show_mtime show_idx\ set collect_variant_from [expr {[currentState commandname] in {avail\ spider} ? {2} : {0}}] set report_via [isEltInReport via] + # modification time is a json-only element, not reported by default + set report_mtime [isEltInReport mtime 0] # prepare list of tag abbreviations that can be substituted and list of # tags whose name should be colored @@ -1540,11 +1545,16 @@ proc reportModules {search_queries header hsgrkey hstyle show_mtime show_idx\ } modulefile - virtual { if {$json} { - ##nagelfar ignore +4 Found constant + # null mtime if modulefile modification time cannot be fetched + set iso_mtime [expr {[lindex $mod_list($elt) 1] ne {} ?\ + "\"[clock format [lindex $mod_list($elt) 1] -format\ + %Y-%m-%dT%H:%M:%S%z]\"" : {null}}] + ##nagelfar ignore +5 Found constant set dispsgr [formatListEltToJsonDisplay $elt type s modulefile\ 1 variants o $vr_list $report_variant symbols a $sym_list\ $report_sym tags a $tag_list $report_tag pathname s [lindex\ - $mod_list($elt) 2] 1 via a $via_list $report_via] + $mod_list($elt) 2] 1 mtime r $iso_mtime $report_mtime via a\ + $via_list $report_via] } elseif {$show_mtime} { set clock_mtime [expr {[lindex $mod_list($elt) 1] ne {} ?\ [clock format [lindex $mod_list($elt) 1] -format {%Y/%m/%d\ diff --git a/tcl/subcmd.tcl.in b/tcl/subcmd.tcl.in index 14139d5b3..9d3dd8198 100644 --- a/tcl/subcmd.tcl.in +++ b/tcl/subcmd.tcl.in @@ -37,12 +37,13 @@ proc cmdModuleList {show_oneperline show_mtime search_match args} { set report_indesym [isEltInReport indesym 0] set report_alias [expr {[isEltInReport alias 0] || [isEltInReport\ provided-alias 0]}] + set fetch_mtime [expr {$show_mtime || [isEltInReport mtime 0]}] # build list of loaded modules and symbolics and aliases if reported set loadedmodlist [list] foreach mod [getEnvLoadedModulePropertyParsedList name] { set modfile [getModulefileFromLoadedModule $mod] - set mtime [expr {$show_mtime && [file exists $modfile] ?\ + set mtime [expr {$fetch_mtime && [file exists $modfile] ?\ [getFileMtime $modfile] : {}}] set mod_list($mod) [list modulefile $mtime $modfile] @@ -1924,6 +1925,7 @@ proc cmdModuleAvail {show_oneperline show_mtime show_filter search_filter\ # elements to include in output set report_modulepath [isEltInReport modulepath] + set fetch_mtime [expr {$show_mtime || [isEltInReport mtime 0]}] # consolidate search filters lappend search_filter $search_match wild @@ -1934,7 +1936,7 @@ proc cmdModuleAvail {show_oneperline show_mtime show_filter search_filter\ inhibitErrorReport # look if aliases have been defined in the global or user-specific rc - array set mod_list [getMatchingAnyModules {} $args $show_mtime\ + array set mod_list [getMatchingAnyModules {} $args $fetch_mtime\ $search_rc_filter $show_filter] if {$report_modulepath} { reportModules $args {global/user modulerc} hi $hstyle $show_mtime 0\ @@ -1942,7 +1944,7 @@ proc cmdModuleAvail {show_oneperline show_mtime show_filter search_filter\ } foreach dir $modpath_list { - set dir_mod_list [getMatchingAnyModules $dir $args $show_mtime\ + set dir_mod_list [getMatchingAnyModules $dir $args $fetch_mtime\ $search_filter $show_filter] if {$report_modulepath} { array unset mod_list diff --git a/testsuite/modules.01-general/120-siteconfig.exp b/testsuite/modules.01-general/120-siteconfig.exp index 22bb4aaf4..47892b09c 100644 --- a/testsuite/modules.01-general/120-siteconfig.exp +++ b/testsuite/modules.01-general/120-siteconfig.exp @@ -775,10 +775,10 @@ lappend ans { "displaySeparatorLine $header $sgrkey $extra"} lappend ans { (procedure "displayElementList" line 19)} lappend ans { invoked from within} lappend ans { "displayElementList $header $hsgrkey $hstyle $one_per_line $show_idx 1 $display_list $len_list $via_list"} -lappend ans { (procedure "reportModules" line 148)} +lappend ans { (procedure "reportModules" line 152)} lappend ans { invoked from within} lappend ans { "reportModules $args $dir mp $hstyle $show_mtime 0 $one_per_line $theader_cols hidden-loaded"} -lappend ans { (procedure "cmdModuleAvail" line 43)} +lappend ans { (procedure "cmdModuleAvail" line 44)} lappend ans { invoked from within} lappend ans { "{*}[formatModuleCommand 1 $command load $tag_list $addpath_pos $show_oneperline $show_mtime $show_filter $search_filter $search_match $dump_state $par..."} lappend ans { (procedure "moduleTop" line 30)} diff --git a/testsuite/modules.70-maint/220-config.exp b/testsuite/modules.70-maint/220-config.exp index 3028e3ace..64c1ed1c2 100644 --- a/testsuite/modules.70-maint/220-config.exp +++ b/testsuite/modules.70-maint/220-config.exp @@ -220,7 +220,7 @@ array set configvalid [list\ advanced_version_spec {0 1}\ auto_handling {0 1}\ avail_indepth {0 1}\ - avail_json_output {modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via}\ + avail_json_output {modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime}\ avail_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec via}\ avail_terse_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec}\ cache_buffer_bytes {4096 1000000}\ @@ -237,7 +237,7 @@ array set configvalid [list\ implicit_default {0 1}\ implicit_requirement {0 1}\ info_extension {0 1}\ - list_json_output {variant alias sym tag hidden via}\ + list_json_output {variant alias sym tag hidden via mtime}\ list_output {header idx variant alias indesym sym tag hidden key}\ list_terse_output {header idx variant alias indesym sym tag hidden key}\ logged_events {auto_eval requested_eval requested_cmd}\ @@ -257,7 +257,7 @@ array set configvalid [list\ silent_shell_debug {0 1}\ source_cache {0 1}\ spider_indepth {0 1}\ - spider_json_output {modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via}\ + spider_json_output {modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime}\ spider_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec via}\ spider_terse_output {modulepath alias provided-alias dirwsym indesym sym tag key hidden variant variantifspec}\ sticky_purge {error warning silent}\ diff --git a/testsuite/modules.70-maint/360-avail_output.exp b/testsuite/modules.70-maint/360-avail_output.exp index a0c7d9db5..4264a0bdf 100644 --- a/testsuite/modules.70-maint/360-avail_output.exp +++ b/testsuite/modules.70-maint/360-avail_output.exp @@ -108,8 +108,8 @@ testouterr_cmd sh {avail --output=idx foo} ERR $tserr_invalid_output_val testouterr_cmd sh {avail --output=header:key foo} ERR $tserr_invalid_output_val # invalid value set on json output mode -set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on avail sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" -set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on avail sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on avail sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on avail sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime (separated by ':')\" \] } \]\n}" testouterr_cmd sh {avail -j --output=foo foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {avail -j --output=sym,alias foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {avail -j -o foo foo} ERR $tserr_invalid_json_o_val @@ -119,6 +119,9 @@ testouterr_cmd sh {avail -j --output=key foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {avail -j --output=sym:indesym foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {avail -j -o header foo} ERR $tserr_invalid_json_o_val testouterr_cmd sh {avail -j -o idx foo} ERR $tserr_invalid_json_o_val +# element accepted on json output mode only +testouterr_cmd sh {avail -o mtime foo} ERR $tserr_invalid_o_val +testouterr_cmd sh {avail --output=mtime foo} ERR $tserr_invalid_output_val # test usage in modulefile set ans [list] @@ -1059,6 +1062,18 @@ setenv_var MODULES_AVAIL_JSON_OUTPUT key:sym testouterr_cmd sh {avail -j foo/1.0} OK $tserr_avail_json unsetenv_var MODULES_AVAIL_JSON_OUTPUT +# mtime element +set foo_mtime [clock format [file mtime $mp/foo/1.0] -format %Y-%m-%dT%H:%M:%S%z] +set tserr_avail_json_mtime "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\", \"mtime\": \"$foo_mtime\"}}" +testouterr_cmd sh {avail -j -o mtime foo/1.0} OK $tserr_avail_json_mtime +testouterr_cmd sh {avail -j --output=mtime foo/1.0} OK $tserr_avail_json_mtime +setenv_var MODULES_AVAIL_JSON_OUTPUT mtime +testouterr_cmd sh {avail -j foo/1.0} OK $tserr_avail_json_mtime +testouterr_cmd sh {avail -j -o +via foo/1.0} OK "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\", \"mtime\": \"$foo_mtime\", \"via\": \[\]}}" +# no modification time on regular/terse output +testouterr_cmd_re sh {avail -t foo/1.0} OK "$mpre:\nfoo/1.0\\(bar\\) " +unsetenv_var MODULES_AVAIL_JSON_OUTPUT + # variant and variantifspec elements setenv_var TESTSUITE_VARIANT 13 set tserr_avail_json_variant "{\"variant/1.0\": { \"name\": \"variant/1.0\", \"type\": \"modulefile\", \"variants\": { \"bar\": \[ \"val1\", \"val2\", \"val3\" \], \"foo\": \[ \"val1\" \] }, \"pathname\": \"$mp/variant/1.0\"}}" diff --git a/testsuite/modules.70-maint/361-list_output.exp b/testsuite/modules.70-maint/361-list_output.exp index 3817533f7..240e6bff7 100644 --- a/testsuite/modules.70-maint/361-list_output.exp +++ b/testsuite/modules.70-maint/361-list_output.exp @@ -83,8 +83,8 @@ testouterr_cmd sh {list -o sym,alias} ERR $tserr_invalid_o_val testouterr_cmd sh {list -o "sym alias"} ERR $tserr_invalid_o_val # invalid value set on json output mode -set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on list sub-command\", \"Allowed elements are: variant alias sym tag hidden via (separated by ':')\" \] } \]\n}" -set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on list sub-command\", \"Allowed elements are: variant alias sym tag hidden via (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on list sub-command\", \"Allowed elements are: variant alias sym tag hidden via mtime (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on list sub-command\", \"Allowed elements are: variant alias sym tag hidden via mtime (separated by ':')\" \] } \]\n}" testouterr_cmd sh {list -j --output=foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {list -j --output=sym,alias} ERR $tserr_invalid_json_output_val testouterr_cmd sh {list -j -o foo} ERR $tserr_invalid_json_o_val @@ -95,6 +95,7 @@ testouterr_cmd sh {list -j --output=sym:idx} ERR $tserr_invalid_json_output_val testouterr_cmd sh {list -j -o key} ERR $tserr_invalid_json_o_val testouterr_cmd sh {list -j -o indesym} ERR $tserr_invalid_json_o_val # element accepted on json output mode only +testouterr_cmd sh {list -o mtime} ERR $tserr_invalid_o_val testouterr_cmd sh {list -t --output=via} ERR $tserr_invalid_output_val testouterr_cmd sh {list -o foo -l} ERR "$error_msgs: Unsupported option '-o' on long output mode" testouterr_cmd sh {ml -o foo} ERR $tserr_invalid_o_val @@ -388,6 +389,20 @@ testouterr_cmd sh {list -j -o tag} OK "{}" testouterr_cmd sh {list -j -o tag:hidden} OK $tserr_list_json_hidden testouterr_cmd sh {list -j --all -o tag} OK $tserr_list_json_hidden unsetenv_var __MODULES_LMTAG + +# mtime element +set foo_mtime [clock format [file mtime $mp/foo/1.0] -format %Y-%m-%dT%H:%M:%S%z] +set tserr_list_json_mtime "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\", \"mtime\": \"$foo_mtime\"}}" +testouterr_cmd sh {list -j -o mtime} OK $tserr_list_json_mtime +setenv_var MODULES_LIST_JSON_OUTPUT mtime +testouterr_cmd sh {list -j} OK $tserr_list_json_mtime +testouterr_cmd sh {list -j -o +via} OK "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\", \"mtime\": \"$foo_mtime\", \"via\": \[\]}}" +# no modification time on regular/terse output +testouterr_cmd sh {list -t} OK $cur_loaded\nfoo/1.0 +# null modification time if modulefile cannot be found +setenv_loaded_module [list foo/1.0] [list $mp/foo/unk] +testouterr_cmd sh {list -j} OK "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/unk\", \"mtime\": null}}" +unsetenv_var MODULES_LIST_JSON_OUTPUT unsetenv_var __MODULES_LMVARIANT unsetenv_var __MODULES_LMALTNAME diff --git a/testsuite/modules.70-maint/362-spider_output.exp b/testsuite/modules.70-maint/362-spider_output.exp index a2aa32b73..f0e7ba98f 100644 --- a/testsuite/modules.70-maint/362-spider_output.exp +++ b/testsuite/modules.70-maint/362-spider_output.exp @@ -94,8 +94,8 @@ testouterr_cmd sh {spider --output=idx foo} ERR $tserr_invalid_output_val testouterr_cmd sh {spider --output=header:key foo} ERR $tserr_invalid_output_val # invalid value set on json output mode -set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on spider sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" -set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on spider sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_output_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '--output' option on spider sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime (separated by ':')\" \] } \]\n}" +set tserr_invalid_json_o_val "{\"errors\": \[\n{ \"severity\": \"ERROR\", \"message\": \[ \"Invalid element in value list for '-o' option on spider sub-command\", \"Allowed elements are: modulepath alias provided-alias dirwsym sym tag hidden variant variantifspec via mtime (separated by ':')\" \] } \]\n}" testouterr_cmd sh {spider -j --output=foo foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {spider -j --output=sym,alias foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {spider -j -o foo foo} ERR $tserr_invalid_json_o_val @@ -105,6 +105,9 @@ testouterr_cmd sh {spider -j --output=key foo} ERR $tserr_invalid_json_output_va testouterr_cmd sh {spider -j --output=sym:indesym foo} ERR $tserr_invalid_json_output_val testouterr_cmd sh {spider -j -o header foo} ERR $tserr_invalid_json_o_val testouterr_cmd sh {spider -j -o idx foo} ERR $tserr_invalid_json_o_val +# element accepted on json output mode only +testouterr_cmd sh {spider -o mtime foo} ERR $tserr_invalid_o_val +testouterr_cmd sh {spider --output=mtime foo} ERR $tserr_invalid_output_val # @@ -1058,6 +1061,15 @@ setenv_var MODULES_SPIDER_JSON_OUTPUT indesym:sym testouterr_cmd sh {spider -j foo/1.0} OK $tserr_spider_json unsetenv_var MODULES_SPIDER_JSON_OUTPUT +# mtime element +set foo_mtime [clock format [file mtime $mp/foo/1.0] -format %Y-%m-%dT%H:%M:%S%z] +testouterr_cmd sh {spider -j -o mtime foo/1.0} OK "{\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\", \"mtime\": \"$foo_mtime\"}}" +setenv_var MODULES_SPIDER_JSON_OUTPUT mtime:modulepath +testouterr_cmd sh {spider -j foo/1.0} OK "{\"$mp\": {\n\"foo/1.0\": { \"name\": \"foo/1.0\", \"type\": \"modulefile\", \"pathname\": \"$mp/foo/1.0\", \"mtime\": \"$foo_mtime\"}\n}}" +# no modification time on regular/terse output +testouterr_cmd_re sh {spider -t foo/1.0} OK "$mpre:\nfoo/1.0\\(bar\\) " +unsetenv_var MODULES_SPIDER_JSON_OUTPUT + # # Cleanup From a2ce773fa619d4156e025428edb2b63e3d9eae59 Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Tue, 3 Nov 2026 07:31:31 +0100 Subject: [PATCH 4/4] doc: describe JSON output configuration in MIGRATING Add a v5.8 section to the migration guide to introduce the JSON output configuration options, the --output support on --json mode, the mtime element and the variant report on avail and spider JSON output. Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Xavier Delaruelle --- .hunspell.en.dic | 1 + MIGRATING.rst | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/.hunspell.en.dic b/.hunspell.en.dic index 720ee12cd..b6b6b4d08 100644 --- a/.hunspell.en.dic +++ b/.hunspell.en.dic @@ -1661,3 +1661,4 @@ otherargv parseModuleCommandArgs DDTHH hhmm +22T21 diff --git a/MIGRATING.rst b/MIGRATING.rst index 0f1fde662..bfe034722 100644 --- a/MIGRATING.rst +++ b/MIGRATING.rst @@ -8,6 +8,66 @@ Modules. It provides an overview of the new features and changed behaviors that will be encountered when upgrading. +v5.8 +---- + +This new version is backward-compatible with previous version 5 release. It +fixes bugs but also introduces new functionalities that are described in this +section. See the :ref:`5.8 release notes<5.8 release notes>` for a complete +list of the changes between Modules v5.7 and v5.8. + +Configuring JSON output content +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The content of the JSON output of :subcmd:`avail`, :subcmd:`list` and +:subcmd:`spider` sub-commands can now be configured, like their regular and +terse outputs. The new :mconfig:`avail_json_output`, +:mconfig:`list_json_output` and :mconfig:`spider_json_output` configuration +options define the elements to report in the JSON document. An element +corresponds either to a key of the module JSON objects (``sym``, ``tag``, +``variant``, ``via``), to a kind of entry to include (``alias``, +``provided-alias``, ``dirwsym``, ``hidden``) or to the grouping of modules by +modulepath (``modulepath``). The :option:`--output`/:option:`-o` command line +switch is now accepted on :option:`--json` output mode to change the reported +content for one command execution. + +By default, the same JSON document than previous versions is produced. When +an element is not enabled, its key is omitted from the JSON objects. Without +the ``modulepath`` element, the top-level JSON object is directly keyed by +module names. + +.. parsed-literal:: + + :ps:`$` module avail --json foo + {"/path/to/modulefiles": { + "foo/1.0": { "name": "foo/1.0", "type": "modulefile", "symbols": [ "default" ], "tags": [], "pathname": "/path/to/modulefiles/foo/1.0", "via": []} + }} + :ps:`$` module avail --json -o sym foo + {"foo/1.0": { "name": "foo/1.0", "type": "modulefile", "symbols": [ "default" ], "pathname": "/path/to/modulefiles/foo/1.0"}} + +The new ``mtime`` element, specific to JSON output mode, reports the +modification time of each modulefile as an ISO 8601 string. It is not part +of the default content as it requires to stat every modulefile when no module +cache file is available. + +.. parsed-literal:: + + :ps:`$` module config avail_json_output +mtime + :ps:`$` module avail --json foo + {"/path/to/modulefiles": { + "foo/1.0": { "name": "foo/1.0", "type": "modulefile", "symbols": [ "default" ], "tags": [], "pathname": "/path/to/modulefiles/foo/1.0", "mtime": "2026-09-22T21:03:28+0200", "via": []} + }} + +Variants are also reported on :subcmd:`avail` and :subcmd:`spider` JSON +output when the ``variant`` or ``variantifspec`` element is enabled. The +possible values of each variant are set as a JSON array. + +.. parsed-literal:: + + :ps:`$` module avail --json -o variant foo + {"foo/1.0": { "name": "foo/1.0", "type": "modulefile", "variants": { "arch": [ "x86_64", "arm64" ], "debug": [ "on", "off" ] }, "pathname": "/path/to/modulefiles/foo/1.0"}} + + v5.7 ----