diff --git a/README.md b/README.md index 813dd14..b24b2d9 100644 --- a/README.md +++ b/README.md @@ -11,3 +11,9 @@ Dependencies for Build: Build the docs locally with `make html` and you'll find the built docs entry point in `_build/html/index.html`. + +## Acknowledgement + +Funded by the European Union + +This work has received funding from the European Union's Horizon Europe research and innovation programme under grant agreement No 101070254 ([CORESENSE](https://coresense.eu)). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the granting authority can be held responsible for them. diff --git a/about/index.rst b/about/index.rst index 65d9cdd..d3b4226 100644 --- a/about/index.rst +++ b/about/index.rst @@ -6,30 +6,31 @@ About and Contact About ***** -CoreSense Project(CoreSense in short) is a project whose objective is to provide blah blah +CoreSense (*A Hybrid Cognitive Architecture for Deep Understanding*) is a four-year research project (October 2022 – September 2026) funded by the European Union's Horizon Europe programme under grant agreement No 101070254. It develops a theory of understanding and awareness for robots and a cognitive architecture that implements it, and it releases the resulting software to the ROS community. -We strive to create an open community and encourage new ROS users and experts alike to collaborate. -However, that can't happen without your issues, pull requests, and support. -We would like to thank here our current and past contributors and maintainers. +The CoreSense consortium: -Our current leadership team includes (add more!!): +- Universidad Politécnica de Madrid (UPM), Spain, coordinator +- Technische Universiteit Delft (TUD), the Netherlands +- Fraunhofer IPA (FhG), Germany +- Universidad Rey Juan Carlos (URJC), Spain +- PAL Robotics (PAL), Spain +- Irish Manufacturing Research (IMR), Ireland +- Czech Technical University in Prague (CVUT), Czech Republic -+------------------+------------------------------------+-------------+----------------+ -| Name | Organization | GitHub ID | Current Role | -+==================+====================================+=============+================+ -| Ricardo Sanz | Universidad Politécnica de Madrid | ricardo_ | Project Leader | -+------------------+------------------------------------+-------------+----------------+ -| Francisco Martín | Rey Juan Carlos University | fmrico_ | WP5 Leader | -+------------------+------------------------------------+-------------+----------------+ - -.. _fmrico: https://github.com/fmrico -.. _ricardo: https://github.com/ricardo +The project website, with news, publications and the public deliverables, is https://coresense.eu. Contact ******* -If you are interested in contacting someone about CoreSense Project please email the project leader. -We intentionally make our emails easy to find. -If your inquiry relates to bugs or open-source feature requests, consider posting a ticket on our GitHub project. -If your inquiry relates to configuration support or private feature development, reach out and we may be able to connect you with -independent consultants or contractors that know this project well. +- Questions about a package, bug reports and feature requests: open an issue in the corresponding repository of the `CoreSense GitHub organisation `_. +- Questions about the project: see the contact information at https://coresense.eu. + +Funding +******* + +.. image:: ../images/eu_funded_en.jpg + :width: 300 + :alt: Funded by the European Union + +CoreSense has received funding from the European Union's Horizon Europe research and innovation programme under grant agreement No 101070254. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the granting authority can be held responsible for them. diff --git a/build_instructions/index.rst b/build_instructions/index.rst index 5c34a9f..0bcf74c 100644 --- a/build_instructions/index.rst +++ b/build_instructions/index.rst @@ -3,32 +3,48 @@ Build and Install ################# -Install -******* +CoreSense software can be installed in three ways. Use binary packages when they exist, and build from source for the rest. -CoreSense install +========================== ============================== ====================================== +Method Requires Available for +========================== ============================== ====================================== +Pixi (conda packages) Pixi, any Linux distribution Packages in the CoreSense channels +``apt`` (Debian packages) Ubuntu with ROS 2 installed Packages in the ROS 2 buildfarm +From source ROS 2 and ``colcon`` All the repositories +========================== ============================== ====================================== +The :ref:`packages` page shows which method is available for each package. -Build -***** +Install with Pixi +***************** -Install ROS ------------ +See :ref:`getting_started`. The CoreSense channels are: -Please install ROS 2 via the usual `build instructions `_ for your desired distribution. +- Jazzy: https://prefix.dev/channels/coresense-jazzy +- Kilted: https://prefix.dev/channels/coresense-kilted -Build CoreSense ---------------- +Install with apt +**************** -Create a new workspace, ``CoreSense_ws``, and clone CoreSense master branch into it and build it. +Install ROS 2 following the `official instructions `_, then install the released packages. For example, for EasyNav on Jazzy: -.. code:: bash +.. code-block:: bash - mkdir -p ~/CoreSense_ws/src - cd ~/CoreSense_ws/src - git clone https://github.com/CoreSenseEU/whatever.git - - cd ~/CoreSense_ws - rosdep install -y -r -q --from-paths src --ignore-src --rosdistro - colcon build --symlink-install + sudo apt install ros-jazzy-easynav ros-jazzy-easynav-simple-planner +Build from source +***************** + +Install ROS 2 following the `official instructions `_. Then create a workspace, clone the repository, install its dependencies and build it. For example, for the CoreSense architecture used in the social testbed: + +.. code-block:: bash + + mkdir -p ~/coresense_ws/src + cd ~/coresense_ws/src + git clone https://github.com/CoreSenseEU/cs4home_architecture.git + cd ~/coresense_ws + rosdep install --from-paths src --ignore-src -r -y + colcon build --symlink-install + source install/setup.bash + +Replace the repository with the one you need. Each repository README describes its specific dependencies and branches. diff --git a/conf.py b/conf.py index 612d387..23525f8 100644 --- a/conf.py +++ b/conf.py @@ -55,7 +55,7 @@ # General information about the project. project = u'CoreSense' -copyright = u'2022' +copyright = u'2022-2026, CoreSense Consortium' author = u'Various' # The version info for the project you're documenting, acts as replacement for @@ -75,12 +75,12 @@ # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. -language = None +language = 'en' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path -exclude_patterns = ['_build','_themes','scripts' ] +exclude_patterns = ['_build','_themes','scripts','extra'] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' @@ -192,3 +192,6 @@ extlinks = {'projectfile': ('https://github.com/CoreSenseEU/%s', 'filepath ')} + +# Extra files copied as-is to the output, e.g. redirects for old URLs. +html_extra_path = ['extra'] diff --git a/demos/docs/dummydemo.rst b/demos/docs/dummydemo.rst deleted file mode 100644 index 92a651d..0000000 --- a/demos/docs/dummydemo.rst +++ /dev/null @@ -1,6 +0,0 @@ -.. _dummydemo: - -Dummy Demo -********** - -TBD diff --git a/demos/index.rst b/demos/index.rst index 8e7f8ae..5ab74ca 100644 --- a/demos/index.rst +++ b/demos/index.rst @@ -3,9 +3,9 @@ Demos ##### -CoreSense demos +Demos that can be run without the real robots: -.. toctree:: - :maxdepth: 1 - - docs/dummydemo.rst +- **Drone inspection in simulation**: the simulation environment of the inspection testbed, with Aerostack2. See :ref:`inspection_testbeds` and `TB2_Panel_Inspection_Simulation `_. +- **Interactive assistant in simulation**: a reduced version of the PAL interactive robot demo, with face detection, a chat interface and simulated objects. See `pal_assistant_demo `_. +- **ROS 2 instrumentation**: video of the virtual drivers and the RViz plugin in :ref:`instrumentation`. +- **Navigation**: EasyNav demos with simulated robots, in https://easynavigation.github.io. diff --git a/design/index.rst b/design/index.rst index 780ee46..429d827 100644 --- a/design/index.rst +++ b/design/index.rst @@ -3,6 +3,44 @@ CoreSense design ################ -CoreSense general design. +CoreSense is a hybrid cognitive architecture that gives robots the capability of *understanding*, the world and themselves, and of *awareness*, understanding in synchrony with perception. The theory behind it is described in the public deliverables `D1.3 Theory of Understanding `_ and `D1.4 Theory of Awareness `_. The terms used in the software follow the `CoreSense Ontology (CSO) `_. -TBD +Architecture subsystems +*********************** + +The architecture has three subsystems: + +- **Runtime system**: a modular software structure, based on ROS 2, used to deploy, monitor and control the modules of the robot. It uses the instrumentation of the ROS 2 platform (see :ref:`instrumentation`). +- **Understanding core**: computes meanings on demand, when another subsystem asks for it, for example to generate an explanation for a user. +- **Awareness system**: a cognitive structure that continuously uses the understanding core on the flow of percepts. When the object of perception is the robot itself, it generates self-awareness. + +The architecture is specified in `D2.2 Specification of the CoreSense Architecture `_, and its software assets are described in `D2.3 CoreSense Architecture `_. + +Cognitive modules +***************** + +A cognitive module is a set of ROS 2 nodes that performs a cognitive function. In the ROS 2 implementation of the architecture, `cs4home_architecture `_, each module has five components, managed through ROS 2 lifecycle transitions: + +- **Afferent**: receives information from the system or the environment. +- **Core**: performs the cognitive function. +- **Efferent**: sends the results to other modules or to the robot. +- **Meta**: provides information about the module. +- **Coupling**: coordinates the module with other modules. + +.. code-block:: text + + information --> Afferent --> Core (cognitive function) --> Efferent --> other modules + ^ ^ + Meta Coupling + +A master/flow mechanism coordinates several cognitive modules. The tutorial :ref:`tutorial_cs4home` shows how to build and run example modules. + +Understanding system +******************** + +The understanding system, `coresense_understanding `_, generates strategies to obtain models with given properties. It combines existing models with model-modification skills, the *engines*. Each engine is a ROS 2 node, annotated with its inputs and outputs, whose skill is wrapped in a behaviour tree. The logic of the understanding core is in `understanding-logic `_, and it uses the Vampire theorem prover through `coresense_vampire `_ and the knowledge base `triplestar_kb `_. + +Engineering toolchain +********************* + +CoreSense systems can be designed with model-based tools. See :ref:`toolchain_introduction`. diff --git a/extra/Toolchain/index.html b/extra/Toolchain/index.html new file mode 100644 index 0000000..7e0a3be --- /dev/null +++ b/extra/Toolchain/index.html @@ -0,0 +1,12 @@ + + + + + CoreSense Toolchain + + + + +

This page has moved to CoreSense Toolchain.

+ + diff --git a/getting_started/index.rst b/getting_started/index.rst index 87ecbdd..a593abc 100644 --- a/getting_started/index.rst +++ b/getting_started/index.rst @@ -3,25 +3,53 @@ Getting Started ############### -This document will take you through the process of installing the |PN| binaries -and using |PN|. +The fastest way to try a released CoreSense package is `Pixi `_. Pixi installs ROS 2 and all the dependencies in a project folder, from the `RoboStack `_ channels and the CoreSense channels. It does not need a system-wide ROS installation, ``rosdep`` or ``colcon``, and it works on any Linux distribution. + +This example installs EasyNav, the CoreSense navigation framework, for ROS 2 Jazzy. .. note:: - See the :ref:`build-instructions` for other situations such as building from source or - working with other types of robots. + See :ref:`build-instructions` to install with ``apt`` on Ubuntu, or to build from source. + +1. Install Pixi (version 0.77 or newer) and open a new terminal: + + .. code-block:: bash + + curl -fsSL https://pixi.sh/install.sh | bash + +2. Create a project that uses the CoreSense and RoboStack channels for Jazzy: + + .. code-block:: bash -Installation -************ + pixi init my_app -c https://prefix.dev/coresense-jazzy \ + -c https://prefix.dev/robostack-jazzy -c conda-forge + cd my_app -1. Install the ROS 2 binary packages as described in the official docs -2. Install the |PN| packages using your operating system's package manager: +3. Add the packages. ``ros-jazzy-ros2run`` provides the ``ros2`` command line tool: .. code-block:: bash - sudo apt install ros--CoreSense-* + pixi add ros-jazzy-easynav ros-jazzy-easynav-simple-planner ros-jazzy-ros2run + +4. Run any ROS 2 command inside the environment. Nothing needs to be sourced: + + .. code-block:: bash + + pixi run ros2 pkg list | grep easynav + pixi shell # or open a shell with the environment activated + +For ROS 2 Kilted, replace ``jazzy`` with ``kilted`` in the channels and in the package names. + +To learn how to use EasyNav, see its documentation at https://easynavigation.github.io. + +Available channels +****************** -Running the Example -******************* +============= ================================================= +Distribution Channel +============= ================================================= +Jazzy (LTS) https://prefix.dev/channels/coresense-jazzy +Kilted https://prefix.dev/channels/coresense-kilted +============= ================================================= -TBD +The channel pages list all the available packages and versions. diff --git a/images/eu_funded_en.jpg b/images/eu_funded_en.jpg new file mode 100644 index 0000000..2ab0c75 Binary files /dev/null and b/images/eu_funded_en.jpg differ diff --git a/index.rst b/index.rst index 73d0e35..1aa69f4 100644 --- a/index.rst +++ b/index.rst @@ -7,26 +7,43 @@ Overview ######## -CoreSense Project (CoreSense in short) is a project... TBD +CoreSense (*A Hybrid Cognitive Architecture for Deep Understanding*) is a Horizon Europe research project (2022–2026) that develops a theory of understanding and awareness for robots, and a cognitive architecture that implements it on top of ROS 2. +This site is the technical documentation of the CoreSense software. It explains how to install the released packages, the design of the architecture, and how to use the cognitive modules, the engineering toolchain and the ROS 2 instrumentation developed in the project. The project website, with news, publications and public deliverables, is at https://coresense.eu. -.. image:: images/funding.png - :width: 400 - :alt: CoreSense Project funding +All the software is open source and available in the `CoreSense GitHub organisation `_. + +Where to start +************** + +- :ref:`getting_started`: install a released CoreSense package in a few minutes, with Pixi or ``apt``. +- :ref:`build-instructions`: all the installation options, including building from source. +- :ref:`design`: the CoreSense architecture. +- :ref:`packages`: the catalogue of CoreSense software, grouped by topic. +- :ref:`tutorials`: step-by-step guides. +- :ref:`testbeds`: the three testbeds where CoreSense has been validated. +- :ref:`contribution_guideline`: how to contribute and how to release a package. + +Funding +******* + +.. image:: images/eu_funded_en.jpg + :width: 300 + :alt: Funded by the European Union + +CoreSense has received funding from the European Union's Horizon Europe research and innovation programme under grant agreement No 101070254. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the granting authority can be held responsible for them. .. toctree:: :hidden: getting_started/index.rst - opensourecommunity/index.rst build_instructions/index.rst design/index.rst + packages/index.rst tutorials/index.rst + instrumentation/index.rst + toolchain/index.rst testbeds/index.rst demos/index.rst - toolchain/index.rst + opensourecommunity/index.rst about/index.rst - instrumentation/index.rst - - - diff --git a/instrumentation/index.rst b/instrumentation/index.rst index 291ef98..03909c1 100644 --- a/instrumentation/index.rst +++ b/instrumentation/index.rst @@ -4,7 +4,7 @@ CoreSense Instrumentation ************************* -.. attention:: This package is part of the CORESENSE prject. And is still under development. any feedback is welcome. +.. note:: This package is part of the CoreSense project. Feedback is welcome in the `issue tracker `_. Monitoring robot behavior in real-world applications often requires tracking multiple nodes, each with its own topics. To simplify this, we developed CoreSense Instrumentation. It allows you to monitor the state of the robot’s systems and manage nodes and topics, including creating or deleting them as needed. @@ -24,7 +24,7 @@ The tool works by creating a virtual driver for each component, as shown in :num :scale: 100 :align: center - Coresense Instrumentation Rviz plugin + CoreSense Instrumentation RViz plugin ---------------- How to use diff --git a/opensourecommunity/index.rst b/opensourecommunity/index.rst index e98509a..5a3bd89 100644 --- a/opensourecommunity/index.rst +++ b/opensourecommunity/index.rst @@ -12,4 +12,14 @@ To streamline the review process, the pull request should have a concise yet inf Once the pull request is submitted, maintainers of the main repository will review it. This review process typically involves evaluating the code against coding standards, verifying functionality, and assessing overall impact. The repository maintainers will then decide to approve, request revisions, or reject the pull request based on these criteria. Upon approval, the pull request will be merged, thereby incorporating the changes into the upstream repository. -When implementing changes, adherence to the `CoreSense Developer’s Guidelines `_ is mandatory. Any code not aligning with these guidelines will be deemed non-compliant and will not be merged. Specifically, for contributions involving ROS code, compliance with ROS testing policies is required. This includes meeting the minimum requirement of unit test coverage, ensuring that all components are thoroughly tested to prevent regression and ensure robustness. Additionally, the code must pass all defined linter checks, which validate code quality, formatting, and adherence to stylistic conventions. +When implementing changes, adherence to the `CoreSense Developer’s Guidelines `_ is mandatory. Any code not aligning with these guidelines will be deemed non-compliant and will not be merged. Specifically, for contributions involving ROS code, compliance with ROS testing policies is required. This includes meeting the minimum requirement of unit test coverage, ensuring that all components are thoroughly tested to prevent regression and ensure robustness. Additionally, the code must pass all defined linter checks, which validate code quality, formatting, and adherence to stylistic conventions. + +Releasing a package +******************* + +Packages are released as CoreSense packages when they reach the quality required by the development guidelines and are useful for the ROS community. The package leader requests the release, the committee of package leaders reviews it, and the package leader publishes it for the newest active ROS 2 distributions: + +- In the ROS 2 buildfarm, with ``bloom``, following the `ROS 2 release guide `_. +- In the CoreSense Pixi channels at prefix.dev, following :ref:`tutorial_pixi_release`. + +Every repository must include a ``LICENSE`` file (Apache 2.0 is recommended) and the EU funding acknowledgement in its ``README.md``. diff --git a/packages/index.rst b/packages/index.rst new file mode 100644 index 0000000..efb626d --- /dev/null +++ b/packages/index.rst @@ -0,0 +1,148 @@ +.. _packages: + +Software catalogue +################## + +All the CoreSense software is open source and hosted in the `CoreSense GitHub organisation `_. This page lists the main repositories, grouped by topic. The *Install* column shows how each one can be installed (see :ref:`build-instructions`): **Pixi**, **apt** or **source**. + +Architecture +************ + +.. list-table:: + :header-rows: 1 + :widths: 30 55 15 + + * - Repository + - Description + - Install + * - `cs4home_architecture `_ + - ROS 2 implementation of the CoreSense architecture: cognitive modules (afferent, core, efferent, meta and coupling components) and flows. + - source + * - `cs4home_examples `_ + - Example cognitive modules built with ``cs4home_architecture``. + - source + * - `cs_functional_module_template `_ + - Template to start a new functional module. + - source + * - `coresense_understanding `_ + - Understanding system: generates strategies to obtain models with given properties. + - source + * - `understanding-logic `_ + - Logic of the understanding core. + - source + * - `coresense_understanding_examples `_ + - Example engines for the understanding system. + - source + * - `decision_system `_ + - CoreSense decision system for ROS 2. + - source + * - `triplestar_kb `_ + - Knowledge base for ROS 2 based on RDF-star and RDF 1.2. + - source + * - `coresense_vampire `_ + - ROS 2 node that wraps the Vampire automated theorem prover. + - source + * - `cso `_ + - CoreSense Ontology, resolvable at https://w3id.org/coresense/cso. + - -- + +Cognitive modules and structures +******************************** + +.. list-table:: + :header-rows: 1 + :widths: 30 55 15 + + * - Repository + - Description + - Install + * - `EasyNavigation `_ and `easynav_plugins `_ + - EasyNav, a plugin-based, real-time navigation framework able to include semantic and awareness representations. Documentation: https://easynavigation.github.io. + - Pixi, apt + * - `risk-awareness-module `_ + - Risk awareness module (RiskAM): real-time risk score of visually navigated robots. See :ref:`tutorial_riskam`. + - source + * - `physics-aware-module `_ + - Physics-aware modelling module based on neuro-evolutionary symbolic regression. + - source + * - `fms `_ + - Cognitive structure to deploy AI foundation models inside CoreSense systems. + - source + * - `cs-ne `_ + - CoreSense Navigation Essential. + - source + * - `cs4home_sound_module `_ + - Cognitive module for sound perception. + - source + * - `cs4home_vision_module `_ + - Cognitive module for visual perception. + - source + * - `cs4home_person_tracker_module `_ + - Person tracking from camera and laser detections. + - source + * - `cs4home-explainability `_ + - Explainability framework based on behaviour-tree status and component evidence. + - source + +Toolchain +********* + +.. list-table:: + :header-rows: 1 + :widths: 30 55 15 + + * - Repository + - Description + - Install + * - `rossdl `_ + - ROS System Definition Language: model-based description of ROS 2 systems and code generation. + - source + * - `RosTooling `_ + - Eclipse-based IDE to model ROS systems. See :ref:`toolchain_introduction`. + - update site + * - `CS_ros2model_TBs `_ + - Models of the testbed systems extracted with the introspection tools. + - -- + * - `SysML2-server-api `_ + - Dockerised local SysML v2 server and API pipeline. + - Docker + +ROS 2 instrumentation +********************* + +.. list-table:: + :header-rows: 1 + :widths: 30 55 15 + + * - Repository + - Description + - Install + * - `coresense_instrumentation `_ + - Virtual drivers to activate, deactivate and monitor data flows, and an RViz plugin. See :ref:`instrumentation`. + - source + +Testbeds +******** + +.. list-table:: + :header-rows: 1 + :widths: 30 55 15 + + * - Repository + - Description + - Install + * - `CoreSense4Home `_ + - CoreSense implementation for RoboCup@Home (social testbed). + - source + * - `cs4home_hri_challenge `_ + - HRI challenge of the social testbed. + - source + * - `tiago_sw `_ + - Open-source software for the PAL TIAGo robot used in the social testbed. + - source, apt + * - `collective_awareness_structure `_ + - Collective awareness for multi-robot systems built with Aerostack2 (inspection testbed). + - source + * - `TB2_Panel_Inspection_Simulation `_ + - Simulation of the drone panel inspection testbed. + - source diff --git a/testbeds/docs/manufacturing_testbed.rst b/testbeds/docs/manufacturing_testbed.rst new file mode 100644 index 0000000..30f4a6b --- /dev/null +++ b/testbeds/docs/manufacturing_testbed.rst @@ -0,0 +1,13 @@ +.. _manufacturing_testbed: + +TB1: Manufacturing Testbed +************************** + +The manufacturing testbed demonstrates augmented flexibility and autonomy of robots in manufacturing cells. It has been developed by IMR and by TU Delft at SAM|XL, and it evaluates how CoreSense copes with intrinsic and extrinsic variability in the manufacturing process. + +Software used in this testbed: + +- `risk-awareness-module `_: risk awareness module. See :ref:`tutorial_riskam`. +- `physics-aware-module `_: physics-aware modelling module. + +The concept and requirements of the testbed are described in the public deliverable `D6.1 Manufacturing Testbed Concept and Requirements Specification `_. diff --git a/testbeds/docs/social_testbed.rst b/testbeds/docs/social_testbed.rst new file mode 100644 index 0000000..587f3d9 --- /dev/null +++ b/testbeds/docs/social_testbed.rst @@ -0,0 +1,19 @@ +.. _social_testbed: + +TB3: Social Testbed +******************* + +The social testbed evaluates CoreSense in human-robot interaction, following the tasks of the RoboCup@Home competition: a PAL TIAGo robot interacts with people in a domestic environment. It has been developed by URJC and PAL Robotics, with a copy of the arena at the University of León (León@Home, a certified European Robotics League testbed), at URJC and at PAL. + +The main demonstrator is the RoboCup@Home HRI Challenge: the robot receives a guest, describes and introduces the person, finds a seat and helps with a bag. It uses the following CoreSense software: + +- `cs4home_architecture `_: ROS 2 implementation of the CoreSense architecture (see :ref:`tutorial_cs4home`). +- `cs4home_hri_challenge `_: coordination of the cognitive modules for the HRI challenge. +- `cs4home_sound_module `_: auditory event detection and sound source localisation. +- `cs4home_vision_module `_: scene understanding and perception of contextual entities. +- `cs4home_person_tracker_module `_: person tracking from camera and laser detections. +- `cs4home-explainability `_: human-understandable explanations when failures, timeouts or unexpected situations occur. +- `CoreSense4Home `_: behaviours and bringup for RoboCup@Home. +- `tiago_sw `_: open-source software of the TIAGo robot. + +The testbed is described in the public deliverables `D8.1 Social Testbed Concept and Requirements Specification `_ and D8.3 Social Testbed Implementation. diff --git a/testbeds/index.rst b/testbeds/index.rst index a52a5a9..1128e32 100644 --- a/testbeds/index.rst +++ b/testbeds/index.rst @@ -1,11 +1,13 @@ .. _testbeds: Testbeds -######### +######## -CoreSense Testbeds +CoreSense has been validated in three testbeds, each one focused on different properties of the robot: flexibility, resilience and explainability. The public deliverables of each testbed are available at https://coresense.eu. .. toctree:: :maxdepth: 1 + docs/manufacturing_testbed.rst docs/inspection_testbed.rst + docs/social_testbed.rst diff --git a/tutorials/docs/cs4home_modules.rst b/tutorials/docs/cs4home_modules.rst new file mode 100644 index 0000000..5776587 --- /dev/null +++ b/tutorials/docs/cs4home_modules.rst @@ -0,0 +1,57 @@ +.. _tutorial_cs4home: + +Cognitive modules with cs4home_architecture +******************************************* + +This tutorial builds and runs the example cognitive modules of `cs4home_examples `_, which use the ROS 2 implementation of the CoreSense architecture, `cs4home_architecture `_. See :ref:`design` for the structure of a cognitive module. + +The repository contains two examples: + +- **Face module**: face identities from ``hri_face_detect`` are converted into knowledge-graph updates. +- **YOLO module**: camera images are processed by YOLO and published as detections. + +Build +===== + +Requires ROS 2 and ``vcstool``. + +.. code-block:: bash + + mkdir -p ~/cs4home_examples_ws/src + cd ~/cs4home_examples_ws/src + git clone https://github.com/CoreSenseEU/cs4home_examples.git + vcs import --recursive < cs4home_examples/thirdparty.repos + cd .. + python3 -m venv --system-site-packages py_deps + source py_deps/bin/activate + pip install -r src/thirdparty/hri_face_detect/requirements.txt + pip install -r src/thirdparty/yolov8_ros/requirements.txt + rosdep install --from-paths src --ignore-src -r -y + colcon build --symlink-install + source install/setup.bash + +Run the YOLO module +=================== + +Start a camera driver, then: + +.. code-block:: bash + + ros2 launch cs4home_simple_project yolo_example.launch.py + +Run the face module +=================== + +Start ``hri_face_detect``, then run the module with its parameters: + +.. code-block:: bash + + ros2 run cs4home_simple_project face_module --ros-args \ + --params-file src/cs4home_examples/cs4home_simple_project/config/params_simple.yaml + +The result can be inspected with the knowledge-graph RQt plugin. + +Create your own module +====================== + +`cs_functional_module_template `_ is a starting point for a new module. More complete examples are the perception modules of the social testbed: `cs4home_sound_module `_ and `cs4home_vision_module `_. diff --git a/tutorials/docs/dummy.rst b/tutorials/docs/dummy.rst deleted file mode 100644 index 99e7595..0000000 --- a/tutorials/docs/dummy.rst +++ /dev/null @@ -1,6 +0,0 @@ -.. _dummy: - -Tutorial Dummy -************** - -TBD diff --git a/tutorials/docs/pixi_release.rst b/tutorials/docs/pixi_release.rst new file mode 100644 index 0000000..4734d7e --- /dev/null +++ b/tutorials/docs/pixi_release.rst @@ -0,0 +1,109 @@ +.. _tutorial_pixi_release: + +Publish a package in the CoreSense Pixi channels +************************************************ + +This tutorial publishes a ROS 2 package in the CoreSense Pixi channels, so that anyone can install it with Pixi on any Linux distribution. It summarises the second exercise of the ROSCon 2026 workshop `Declarative ROS workspaces with Pixi `_, and it is also part of the `CoreSense/ROS Development Guidelines (D5.1) `_. + +The examples use a package called ``my_package`` and ROS 2 Jazzy. For Kilted, replace ``jazzy`` with ``kilted`` everywhere. + +1. Install Pixi +=============== + +.. code-block:: bash + + curl -fsSL https://pixi.sh/install.sh | bash + +2. Add a package manifest +========================= + +Put a ``pixi.toml`` next to the ``package.xml`` of each package, and another one at the root of the repository for the workspace: + +.. code-block:: text + + my_repository/ + pixi.toml <- workspace manifest + my_package/ + package.xml + CMakeLists.txt + pixi.toml <- package manifest + +The package manifest says which build backend to use and that the package may be published. Everything else, including name, version and dependencies, comes from ``package.xml``: + +.. code-block:: toml + + [package] + publish = true + + [package.build.backend] + name = "pixi-build-ros" + workspace = true + +For ``ros2 run`` to find an executable, install it in ``lib/``, as in any ROS 2 package. + +3. Add the workspace manifest +============================= + +.. code-block:: toml + + [workspace] + name = "my_repository" + channels = ["https://prefix.dev/coresense-jazzy", + "https://prefix.dev/robostack-jazzy", + "conda-forge"] + platforms = ["linux-64"] + preview = ["pixi-build"] + + [workspace.dependencies] + pixi-build-ros = ">=0.7.5" + + [dependencies] + ros-jazzy-my-package = { path = "my_package" } + ros-jazzy-ros2run = "*" + + [tasks] + start = "ros2 run my_package my_node" + +The conda name of ``my_package`` is ``ros-jazzy-my-package``: distribution prefix, and hyphens instead of underscores. + +4. Build and test locally +========================= + +.. code-block:: bash + + pixi install # builds my_package and installs it in the environment + pixi run start # runs the node, no sourcing needed + +If the build fails, the usual reason is a dependency in ``package.xml`` with a different name in RoboStack. It can be mapped with the ``extra-package-mappings`` option of `pixi-build-ros `_. + +5. Publish to a local folder first +================================== + +.. code-block:: bash + + pixi publish --target-channel ./output + find output -name "*.conda" + +6. Publish to the CoreSense channel +=================================== + +You need a prefix.dev account with write permission on the CoreSense channels. Ask the WP5 leader to add your account, create an API key in your prefix.dev account settings, and then run: + +.. code-block:: bash + + pixi auth login prefix.dev --token + pixi publish --dry-run --target-channel https://prefix.dev/coresense-jazzy + pixi publish --target-channel https://prefix.dev/coresense-jazzy + +A version that already exists in the channel is skipped, so increase the version in ``package.xml`` for every release. Packages are built for the platform of the machine where Pixi runs. + +7. Install the published package +================================ + +.. code-block:: bash + + pixi init my_app -c https://prefix.dev/coresense-jazzy \ + -c https://prefix.dev/robostack-jazzy -c conda-forge + cd my_app + pixi add ros-jazzy-my-package ros-jazzy-ros2run + pixi run ros2 run my_package my_node diff --git a/tutorials/docs/riskam.rst b/tutorials/docs/riskam.rst new file mode 100644 index 0000000..e525efd --- /dev/null +++ b/tutorials/docs/riskam.rst @@ -0,0 +1,67 @@ +.. _tutorial_riskam: + +Risk awareness with RiskAM +************************** + +The Risk Awareness Module (RiskAM), in `risk-awareness-module `_, computes in real time a risk score between 0 and 1 for visually navigated robots, considering the risk to humans around the robot. It combines proximity, gaze, position and approach sub-scores computed from an RGB-D camera. + +Requirements +============ + +- An RGB camera and an absolute depth image in metres. The defaults are calibrated for Intel RealSense D4xx cameras. +- Optionally, the robot velocity (``cmd_vel``) and person tracking, to enable the path-aware and approach sub-scores. +- ROS 2. It is tested on Rolling. + +Build +===== + +.. code-block:: bash + + cd ~/ros2_ws/src + git clone https://github.com/CoreSenseEU/risk-awareness-module.git + cd .. + colcon build --packages-select riskam riskam_ros riskam_msgs riskam_bringup + source install/setup.bash + +Run +=== + +.. code-block:: bash + + ros2 run riskam_ros riskam_node.py --ros-args -p camera_topic:=/your/color/topic + +or, to launch the node together with the data logger: + +.. code-block:: bash + + ros2 launch riskam_bringup riskam.launch.py run_logger:=true + +The parameters are in ``riskam_bringup/config/riskam_config.yml``. The main ones are: + +.. list-table:: + :header-rows: 1 + + * - Parameter + - Default + - Purpose + * - ``camera_topic`` + - ``/camera/camera/color/image_raw`` + - RGB input + * - ``depth_topic`` + - ``/camera/camera/depth/image_rect_raw`` + - Depth input + * - ``cmd_vel_topic`` + - ``/cmd_vel`` + - Robot velocity (optional) + * - ``d_safe`` + - ``1.5`` m + - Safety distance + +Output +====== + +- ``/riskam/risk_score`` (``riskam_msgs/FloatStamped``): risk of the scene, between 0 and 1. +- ``/riskam/annotated_image`` (``sensor_msgs/Image``): visualisation overlay. +- ``/riskam/diagnostics`` (``diagnostic_msgs/DiagnosticArray``): timing and status of each sub-score. + +The full reference of parameters and topics is in the `repository documentation `_. diff --git a/tutorials/index.rst b/tutorials/index.rst index ff78d90..b58a27b 100644 --- a/tutorials/index.rst +++ b/tutorials/index.rst @@ -3,9 +3,17 @@ Tutorials ######### -CoreSense Tutorials +Step-by-step guides to use and extend CoreSense. .. toctree:: :maxdepth: 1 - docs/dummy.rst + docs/cs4home_modules.rst + docs/riskam.rst + docs/pixi_release.rst + +More guides: + +- ROS 2 instrumentation: :ref:`instrumentation`. +- Modelling ROS systems with the toolchain: `RosTooling tutorials `_. +- Navigation with EasyNav: https://easynavigation.github.io.