Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkgs-lib.formats.xml: init #342633

Merged
merged 1 commit into from
Jan 13, 2025
+102 −0
Merged

pkgs-lib.formats.xml: init #342633

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions nixos/doc/manual/development/settings-options.section.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,21 @@ have a predefined type and string generator already declared under
and returning a set with TOML-specific attributes `type` and
`generate` as specified [below](#pkgs-formats-result).

`pkgs.formats.xml` { format ? "badgerfish", withHeader ? true}

: A function taking an attribute set with values
and returning a set with XML-specific attributes `type` and
`generate` as specified [below](#pkgs-formats-result).

`format`

: Input format. Because XML can not be translated one-to-one, we have to use intermediate formats. Possible values:
- `"badgerfish"`: Uses [badgerfish](http://www.sklar.com/badgerfish/) conversion.

`withHeader`

: Outputs the xml with header.

`pkgs.formats.cdn` { }

: A function taking an empty attribute set (for future extensibility)
Expand Down
60 changes: 60 additions & 0 deletions pkgs/pkgs-lib/formats.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -577,4 +577,64 @@ rec {
'') {};
};

xml =
{
format ? "badgerfish",
withHeader ? true,
}:
if format == "badgerfish" then
{
type = let
valueType = nullOr (oneOf [
bool
int
float
str
path
(attrsOf valueType)
(listOf valueType)
]) // {
description = "XML value";
};
in valueType;

generate =
name: value:
pkgs.callPackage (
{
runCommand,
python3,
libxml2Python,
}:
runCommand name
{
nativeBuildInputs = [
python3
python3.pkgs.xmltodict
libxml2Python
];
value = builtins.toJSON value;
pythonGen = ''
import json
import os
import xmltodict

with open(os.environ["valuePath"], "r") as f:
print(xmltodict.unparse(json.load(f), full_document=${toString withHeader}, pretty=True, indent=" " * 2))
'';
passAsFile = [
"value"
"pythonGen"
];
preferLocalBuild = true;
}
''
python3 "$pythonGenPath" > $out
xmllint $out > /dev/null
''
) { };
}
else
throw "pkgs.formats.xml: Unknown format: ${format}";

}
27 changes: 27 additions & 0 deletions pkgs/pkgs-lib/tests/formats.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -644,4 +644,31 @@ in runBuildTests {
'';
};

badgerfishToXmlGenerate = shouldPass {
format = formats.xml { };
input = {
root = {
"@id" = "123";
"@class" = "example";
child1 = {
"@name" = "child1Name";
"#text" = "text node";
};
child2 = {
grandchild = "This is a grandchild text node.";
};
nulltest = null;
};
};
expected = ''
<?xml version="1.0" encoding="utf-8"?>
<root class="example" id="123">
<child1 name="child1Name">text node</child1>
<child2>
<grandchild>This is a grandchild text node.</grandchild>
</child2>
<nulltest></nulltest>
</root>
'';
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would namespaces look?

Does definition merging work as expected?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it follows: https://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html:

doesn't take into consideration the following:

  • XML declaration
  • processing instructions
  • explicit handling of namespace declarations
  • XML comments

i found: https://github.com/martinblech/xmltodict/blob/master/README.md#namespace-support

Copy link
Contributor Author

@Stunkymonkey Stunkymonkey Sep 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

namespaces are "mainly problematic" for parsing the other way around. it can be achieved via:

{
  "RSS": {
    "@xmlns:jwplayer": "http://support.jwplayer.com/customer/portal/articles/1403635-media-format-reference#feeds";
    "@version": "2.0";
    "Channel": {
...

to create:

<?xml version="1.0" encoding="utf-16"?>
<RSS xmlns:jwplayer="http://support.jwplayer.com/customer/portal/articles/1403635-media-format-reference#feeds" version="2.0">
  <Channel>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, makes sense.

Handling foreign input while programming with hardcoded prefixes is basically wrong; an XML document should behave exactly the same under prefix renaming, but a lot of software requires certain prefixes, which is - in principle - wrong.
However, we will have to cater to such applications, and users who are kinda ok with such requirements, so that's just that.

Treating any node as URI + tag instead of prefix + tag makes any XML processing more robust (for an option type, the processing means implementing a merge operation for the XML representation), so that may be worth considering, but it seems that we'd end up writing a small library for that purpose; a bit of a research project.

So for now I think we only need to point out in the docs that the type deals with the concrete syntax of the XML, and all responsibility is on the user when it comes to the correctness of any use of namespace features.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this will not be the "perfect/best" xml-exporter for nix, I would suggest to get started, but name it "specially" (not xml as we both agreed above.).

Surely there are a lot of things to improve, but in the end we should enable other users to enable new NixOS-options. Migrating this to other formats seems tedious (manual work), but doable.

Documentation is currently missing completely, because I do not know where to put it. Any suggestions where to put it?

}