Move DB requests to broker

See merge request !240

Comment-out seconds attempt to ingest dataset in the first attempt failed because if the duplicate error.

build: use same Python interpreter in all CMake scripts

See merge request asapo/asapo!230

These modules were introduced in CMake 3.12 as mentioned on
https://cmake.org/cmake/help/git-stage/release/3.12.html#modules. So
these modules are not needed in `ASAP::O` code base except when building
on Ubuntu 18.04 that contains CMake 3.10. As a workaround, the CI job
`build-packages-ubuntu1804` defines the environment variable
`Python_EXECUTABLE` so that CMake does not need the `FindPython.cmake`
module to find the Python interpreter.

Providing the value of `Python_EXECUTABLE` using an environmnet variable
instead of a CMake argument can simplify calls to CMake:

```sh
cmake .. -DENABLE_LIBFABRIC=ON -DCMAKE_BUILD_TYPE="Release" \
  -DBUILD_CLIENTS_ONLY=ON -DBUILD_PYTHON_PACKAGES=source \
  -DNUMPY_VERSION=$NUMPY_VERSION -DPython_EXECUTABLE=python3

export Python_EXECUTABLE=python3
cmake .. -DENABLE_LIBFABRIC=ON -DCMAKE_BUILD_TYPE="Release" \
  -DBUILD_CLIENTS_ONLY=ON -DBUILD_PYTHON_PACKAGES=source \
  -DNUMPY_VERSION=$NUMPY_VERSION
```

This avoids using different Python interpreters for different tasks and
therefore avoid surprises.

Note: CMake files for Windows were not fixed because no Windows
CI runner checks that `ASAP::O` compiles on Windows.

test: reduce waiting time when locally starting `ASAP::O` services

See merge request asapo/asapo!239

build: disable optimizations when building `go` code in Debug mode

See merge request asapo/asapo!238

doc: remove `data_nbytes` from Python Producer client documentation

See merge request asapo/asapo!235

This commit makes `run_asapo.sh` monitor the status of the services
started using `supervisord` so that it exits when all services are
running instead of waiting for 20 seconds.

Debugging Go code built without optimizations is easier, as mentioned on
https://go.dev/doc/gdb#Introduction:

> When you compile and link your Go programs with the gc toolchain on
> Linux, macOS, FreeBSD or NetBSD, the resulting binaries contain DWARFv4
> debugging information that recent versions (≥7.5) of the GDB debugger
> can use to inspect a live process or a core dump.
>
> Pass the '-w' flag to the linker to omit the debug information (for
> example, go build -ldflags=-w prog.go).
>
> The code generated by the gc compiler includes inlining of function
> invocations and registerization of variables. These optimizations can
> sometimes make debugging with gdb harder. If you find that you need to
> disable these optimizations, build your program using go build
> -gcflags=all="-N -l".
>
> If you want to use gdb to inspect a core dump, you can trigger a dump
> on a program crash, on systems that permit it, by setting
> GOTRACEBACK=crash in the environment (see the runtime package
> documentation for more info).

... by making `data_nbytes` a C function instead of a Python function.
See
https://notes-on-cython.readthedocs.io/en/latest/function_declarations.html
for the difference between `def` and `cdef` in Cython code.

ci: run tests of Python client in CI before deploying containers and packages

See merge request asapo/asapo!237

build: add `raw-loader` dependency for building `docs/site`

See merge request asapo/asapo!236

The Python tests now run in the  CI job `test-services-linux-debug`.
These tests need a new token because using the token contained in
`standalone_token.txt` was making the call `consumer.get_stream_list()` fail:

```
>>> consumer.get_stream_list()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "asapo_consumer.pyx", line 323, in asapo_consumer.PyConsumer.get_stream_list
  File "asapo_consumer.pyx", line 102, in asapo_consumer.throw_exception
asapo_consumer.AsapoWrongInputError: error: wrong input, message: authorizer rejected to authorize: wrong or expired JWT token, details: api:/v0.6/beamtime/asapo_test/source_20240611T_103617/0/streams, host:localhost:8413
```

Instead, the tests use a new token contained in `local_dev_token.txt`
that was generated using PyJWT 2.8.0:

```python
import jwt

key = "12ljzgneasfd"
beamtimetext="asapo_test"
new_token=jwt.encode({
    "exp": 9571710216, #FIXME: will break on 25.04.2273
    "jti": "wtfisthis",
    "sub": "bt_"+beamtimetext,
    "ExtraClaims": {
        "AccessTypes": [
            "write",
            "writeraw",
            "read",
        ],
    },
}, key, algorithm="HS256")
print(new_token)  # This prints the value of the token to put into `local_dev_token.txt`

print(jwt.decode(new_token, key, algorithms=["HS256"]))
```

Diana Rueda authored 9 months ago and Marc-Olivier Andrez committed 9 months ago

\+ Remove `npm install raw-loader` step from CI job `pages`

Without the `raw-loader` package, building `docs/site` by executing the following command was failing:

```sh
$ cd docs/site && npm install
$ yarn build
...
[ERROR] Docusaurus server-side rendering could not render static page with path /docs/21.06.0/.
...
Error: Cannot find module './versioned_examples/version-21.06.0/start_asapo_socket.sh'
...
[WARNING] {"moduleIdentifier":"/home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/docs/site|sync|/(\\.sh%7C\\.py%7C\\.cpp%7C\\.c%7C\\.txt%7CMakefile)$/","moduleName":"././ sync (\\.sh%7C\\.py%7C\\.cpp%7C\\.c%7C\\.txt%7CMakefile)$","loc":"./CMakeLists.txt","message":"Module not found: Error: Can't resolve 'raw-loader' in '/home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/docs/site'","compilerPath":"client"}
...
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
```

feat: show given and expected versions when using incorrect version

See merge request !227

New error message:

```txt
Version v0.1 is not supported. Expected version v0.0.
```

Add file `.git-blame-ignore-revs` to ignore commits when running `git blame`

See merge request asapo/asapo!233

build: enable compiling tests not in debug mode (and without code coverage)

See merge request asapo/asapo!225

Below is the type of compilation error I was getting:

```
$ cmake -G Ninja ../.. -DPython_EXECUTABLE=python3.7 -DBUILD_TESTS=ON -DENABLE_COVERAGE=ON -DBUILD_CPP_DOCS=ON -DBUILD_PYTHON_DOCS=ON
...
$ ninja
[54/174] Building CXX object producer/api/cpp/CMakeFiles/test-asapo-producer.dir/unittests/test_producer_impl.cpp.o
FAILED: producer/api/cpp/CMakeFiles/test-asapo-producer.dir/unittests/test_producer_impl.cpp.o
ccache /usr/bin/c++  -I/home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/producer/api/cpp/include -I/home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/producer/api/cpp/../c/include -I/home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/common/cpp/include -g -O0 --coverage -fprofile-arcs -ftest-coverage -Wall -Wextra -pedantic -Wconversion -Wno-missing-field-initializers -fPIC -DGTEST_HAS_PTHREAD=1 -std=gnu++11 -MD -MT producer/api/cpp/CMakeFiles/test-asapo-producer.dir/unittests/test_producer_impl.cpp.o -MF producer/api/cpp/CMakeFiles/test-asapo-producer.dir/unittests/test_producer_impl.cpp.o.d -o producer/api/cpp/CMakeFiles/test-asapo-producer.dir/unittests/test_producer_impl.cpp.o -c /home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/producer/api/cpp/unittests/test_producer_impl.cpp
In file included from /home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/producer/api/cpp/unittests/test_producer_impl.cpp:15:
/home/marc-olivier/Documents/desy/dev/gitlab.desy.de/asapo/asapo/producer/api/cpp/unittests/mocking.h:20:14: error: ‘uint64_t asapo::MockDiscoveryService::UpdateFrequency()’ marked ‘override’, but does not override
   20 |     uint64_t UpdateFrequency() override {
      |              ^~~~~~~~~~~~~~~
```

- .vscode/
- .ccache/
- .venv/
- venv/
- __pycache__/
- linux_packages/

style: ensure that files terminate with empty new line using `pre-commit`

See merge request asapo/asapo!232

As mentioned on
https://stackoverflow.com/questions/729692/why-should-text-files-end-with-a-newline
and
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206,
POSIX defines a line as

> A sequence of zero or more non- <newline> characters plus a terminating <newline>
> character.

When a file does not terminate with an empty line, `Git` commands such
as `git diff` show the message `No newline at end of file`.

This commit adds a CI job `pre-commit` to enforce the formatting.

build: replace `distutils` with `setuptools`

See merge request !231

build: fix compilation errors when compiling with GCC 11

See merge request !229