Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 33 additions & 15 deletions doc/source/format_intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -245,13 +245,18 @@ author to assert some invalid configurations.

.. _format_directives_list_prepend:

(<) List Prepend
(<) Prepend
~~~~~~~~~~~~~~~~
Indicates that the list should be prepended to the target list,
instead of the default behavior which is to replace the target list.
Indicates that the value should be prepended to the target value,
instead of the default behaviour which is to replace the target value.

When applying to a list, it will insert the new list items before the target list items.
When applying to a string, it will prefix the new string onto the target string.

**Example:**

For lists:
Comment thread
aleixpol marked this conversation as resolved.
Resolved

.. code:: yaml

config:
Expand All @@ -264,10 +269,24 @@ instead of the default behavior which is to replace the target list.

.. _format_directives_list_append:

(>) List Append
For strings:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an example for prepend, so it should be above the _format_directives_list_append marker (and I think the _format_directives_string_append marker is unused.


.. code:: yaml

string-property:
(<): "start"

.. _format_directives_string_append:

(>) Append
~~~~~~~~~~~~~~~
Indicates that the list should be appended to the target list, instead
of the default behavior which is to replace the target list.
Indicates that the values should be appended to its target, instead
of the default behavior which is to replace it.


When applying to a list, it will place the new list items at the end of the list.
When applying to a string, it will place the string value as a postfix.


**Example:**

Expand All @@ -285,21 +304,20 @@ of the default behavior which is to replace the target list.

.. _format_directives_list_overwrite:

(=) List Overwrite
(=) Overwrite
~~~~~~~~~~~~~~~~~~
Indicates that the list should be overwritten completely.
Indicates that the value should be overwritten completely.

This exists mostly for completeness, and we recommend using literal
lists most of the time instead of list overwrite directives when the
intent is to overwrite a list.
This exists mostly for completeness, and we recommend using literals
instead of the overwrite directive when the intent is to overwrite.

This has the same behavior as a literal list, except that an
This has the same behavior as a literal, except that an
error will be triggered in the case that there is no underlying
list to overwrite; whereas a literal list will simply create a new
list.
property to overwrite; whereas a literal entry will simply create a new
property.

The added error protection can be useful when intentionally
overwriting a list in an element's *public data*, which is mostly
overwriting a value in an element's *public data*, which is mostly
free form and not validated.


Expand Down
1 change: 1 addition & 0 deletions src/buildstream/node.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ cdef class MappingNode(Node):
# Protected Methods
cdef void _compose_on_composite_dict(self, MappingNode target)
cdef void _compose_on_list(self, SequenceNode target)
cdef void _compose_on_string(self, ScalarNode target)

# Private Methods
cdef void __composite(self, MappingNode target, list path) except *
Expand Down
23 changes: 23 additions & 0 deletions src/buildstream/node.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -977,6 +977,9 @@ cdef class MappingNode(Node):
if type(target_value) is SequenceNode:
# Composite list composes into a list
self._compose_on_list(target_value)
elif isinstance((<ScalarNode> target_value).value, str):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this require a type check before the cast to ScalarNode?

# Composite list composes into a string
self._compose_on_string(target_value)
elif target_value._is_composite_list():
# Composite list merges into composite list
self._compose_on_composite_dict(target_value)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_compose_on_composite_dict only knows about list, not string composition. Couldn't this be hit via multiple includes? It may be acceptable to start with this as known limitation, if there is a reasonable error.

Expand Down Expand Up @@ -1016,6 +1019,26 @@ cdef class MappingNode(Node):
if suffix is not None:
target.value.extend(suffix.value)

# _compose_on_string(target)
#
# Compose the current node on the given string.
#
# Args:
# target (.SequenceNode): sequence on which to compose the current composite dict

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Not a SequenceNode

#
cdef void _compose_on_string(self, ScalarNode target):
cdef ScalarNode clobber = self.value.get("(=)")
cdef ScalarNode prefix = self.value.get("(<)")
cdef ScalarNode suffix = self.value.get("(>)")

if clobber is not None:
target.value.clear()
target.value.extend(clobber.value)
Comment on lines +1035 to +1036

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right for strings. I'd expect a simple assignment to work fine, though.

Suggested change
target.value.clear()
target.value.extend(clobber.value)
target.value = clobber.value

if prefix is not None:
target.value = prefix.value + target.value
if suffix is not None:
target.value += suffix.value

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As ScalarNode strips whitespace from the start and end of strings (even if they're quoted), it won't be possible to append e.g. " -foo" to a flags variable, as far as I can tell. Not sure what the best solution is for this, especially considering backwards compatibility, but this might block real use cases.


# _compose_on_composite_dict(target)
#
# Compose the current node on the given composite dict.
Expand Down
14 changes: 14 additions & 0 deletions tests/internals/yaml.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,20 @@ def test_composite_preserve_originals(datafiles):
assert orig_extra.get_str("old") == "new"


@pytest.mark.datafiles(os.path.join(DATA_DIR))
def test_composite_strings(datafiles):

filename = os.path.join(datafiles, "basics.yaml")
overlayfile = os.path.join(datafiles, "stringappend.yaml")

base = _yaml.load(filename, shortname=None)
overlay = _yaml.load(overlayfile, shortname=None)
base_copy = base.clone()
overlay._composite(base_copy)

assert base_copy.get_str("description") == ("incoming potato(The vehicle of choice for rainbow travel)potato")


# Tests for list composition
#
# Each test composits a filename on top of basics.yaml, and tests
Expand Down
3 changes: 3 additions & 0 deletions tests/internals/yaml/stringappend.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
description:
(>): ")potato"
(<): "incoming potato("
Loading