Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions .github/workflows/tests-dev.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
name: Tests (dev)

# The dev branch tracks TA-Lib C's dev branch, which has no release yet, so this
# builds the C library from source and the two dev lines move together. master
# is tests.yml, which installs the published release; the steps after the C
# build are that file's, and are meant to stay in step with it.
# The dev branch tracks TA-Lib C's dev branch, which has no release yet, so
# setup.py builds the C library from the dev tarball (TALIB_C_URL) and the two
# dev lines move together. master is tests.yml, which builds the published
# release; the steps after this are that file's, and are meant to stay in step
# with it.

on:
push:
Expand All @@ -18,9 +19,7 @@ jobs:
matrix:
python-version: ["3.11", "3.12", "3.13", "3.14", "3.14t", "3.15", "3.15t"]
env:
TA_INCLUDE_PATH: ${{ github.workspace }}/ta-lib-c/include
TA_LIBRARY_PATH: ${{ github.workspace }}/ta-lib-c/lib
LD_LIBRARY_PATH: ${{ github.workspace }}/ta-lib-c/lib
TALIB_C_URL: https://github.com/TA-Lib/ta-lib/archive/refs/heads/dev.tar.gz
steps:
- uses: actions/checkout@v3

Expand Down Expand Up @@ -70,7 +69,7 @@ jobs:
else
pip install -r requirements_test.txt
fi
pip install flake8
pip install flake8 cmake

- name: Build cython modules in-place
run: |
Expand Down
15 changes: 1 addition & 14 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,6 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: TA-Lib C version
id: talib_c
run: |
set -euo pipefail
V=$(sed -n 's/^ TALIB_C_VER: *//p' .github/workflows/wheels.yml)
[ -n "$V" ] || { echo "::error::no TALIB_C_VER in wheels.yml"; exit 1; }
echo "version=$V" >> "$GITHUB_OUTPUT"

- name: Set up TA-Lib
uses: TA-Lib/setup-ta-lib@v1
with:
version: ${{ steps.talib_c.outputs.version }}

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with:
Expand All @@ -45,7 +32,7 @@ jobs:
else
pip install -r requirements_test.txt
fi
pip install flake8
pip install flake8 cmake

- name: Build cython modules in-place
run: |
Expand Down
58 changes: 58 additions & 0 deletions .github/workflows/tsan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: TSAN

# Runs the test suite under ThreadSanitizer on free-threaded CPython, using the
# prebuilt TSAN images (CPython and NumPy both built with -fsanitize=thread).

on:
push:
branches: [ master, 'dev-*' ]
pull_request:
branches: [ master, 'dev-*' ]
workflow_dispatch:

jobs:
tsan:
name: TSAN (${{ matrix.python }})
runs-on: ubuntu-24.04
timeout-minutes: 60
continue-on-error: ${{ matrix.experimental }}
strategy:
fail-fast: false
matrix:
include:
- python: "3.14t"
image: ghcr.io/nascheme/numpy-tsan:3.14t
cpython_ref: "3.14"
experimental: false
- python: "3.15t"
image: ghcr.io/nascheme/numpy-tsan:3.15t
cpython_ref: main
experimental: false
steps:
- uses: actions/checkout@v4

- name: Lower ASLR entropy for TSAN
run: sudo sysctl vm.mmap_rnd_bits=28

- name: Build with TSAN and run the tests
env:
IMAGE: ${{ matrix.image }}
CPYTHON_REF: ${{ matrix.cpython_ref }}
run: |
docker run --rm -i -v "$PWD":/src -w /src -e CPYTHON_REF "$IMAGE" bash -euo pipefail -s <<'EOF'
python -c "import sys, sysconfig; print(sys.version); print(sysconfig.get_config_var('CONFIG_ARGS'))"
python -m pip install -r requirements.txt pytest setuptools cmake

TA_TSAN=1 python setup.py build_ext --inplace
grep -q __tsan talib/_ta_lib*.so

python -W error -c "import sys, talib; assert not sys._is_gil_enabled()"

python -c "import os, urllib.request; urllib.request.urlretrieve('https://raw.githubusercontent.com/python/cpython/' + os.environ['CPYTHON_REF'] + '/Tools/tsan/suppressions_free_threading.txt', '/tmp/tsan_suppressions.txt')"
export TSAN_OPTIONS="suppressions=/tmp/tsan_suppressions.txt handle_segv=0 second_deadlock_stack=1"

PYTHONPATH=. python -m pytest -s -ra tests \
--ignore=tests/test_pandas.py --ignore=tests/test_polars.py \
--deselect tests/test_abstract.py::test_pandas \
--deselect tests/test_abstract.py::test_pandas_series \
--deselect tests/test_stream.py::test_stream_rejects_mixed_series
57 changes: 8 additions & 49 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ on:
workflow_dispatch:

env:
TALIB_C_VER: 0.8.1
PIP_NO_VERIFY: 0
PIP_VERBOSE: 1
CIBW_BEFORE_BUILD: pip install -U setuptools Cython wheel meson-python ninja && pip install -U numpy
TA_FORCE_BUNDLED: 1
CIBW_BEFORE_BUILD: pip install -U setuptools Cython wheel cmake && pip install -U numpy
CIBW_SKIP: "pp* cp39-* cp310-* cp313t*"

jobs:
Expand All @@ -26,11 +26,9 @@ jobs:
- name: Build wheels
uses: pypa/cibuildwheel@v4.2.1
env:
CIBW_BEFORE_ALL: |
bash ./tools/build_talib_linux.sh
CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28
CIBW_MUSLLINUX_X86_64_IMAGE: musllinux_1_2
CIBW_ENVIRONMENT_PASS_LINUX: TALIB_C_VER CFLAGS
CIBW_ENVIRONMENT_PASS_LINUX: CFLAGS TA_FORCE_BUNDLED
CFLAGS: -D_GNU_SOURCE
CIBW_TEST_SKIP: "cp*-musllinux*"
CIBW_TEST_COMMAND: >
Expand All @@ -39,12 +37,8 @@ jobs:
CIBW_ARCHS_LINUX: x86_64
CIBW_BUILD_FRONTEND: build
CIBW_ENVIRONMENT_LINUX : >
TA_LIBRARY_PATH="ta-lib-install/lib"
TA_INCLUDE_PATH="ta-lib-install/include"
PIP_NO_BUILD_ISOLATION=false
CIBW_REPAIR_WHEEL_COMMAND_LINUX: |
export LD_LIBRARY_PATH="/project/ta-lib-install/lib:$LD_LIBRARY_PATH"
auditwheel repair -w {dest_dir} {wheel}
TA_BUNDLE_CACHE=/tmp/ta-lib-cache
- name: Upload wheels
uses: actions/upload-artifact@v7
with:
Expand All @@ -62,11 +56,9 @@ jobs:
- name: Build wheels
uses: pypa/cibuildwheel@v4.2.1
env:
CIBW_BEFORE_ALL: |
bash ./tools/build_talib_linux.sh
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux_2_28
CIBW_MUSLLINUX_AARCH64_IMAGE: musllinux_1_2
CIBW_ENVIRONMENT_PASS_LINUX: TALIB_C_VER CFLAGS
CIBW_ENVIRONMENT_PASS_LINUX: CFLAGS TA_FORCE_BUNDLED
CFLAGS: -D_GNU_SOURCE
CIBW_TEST_SKIP: "cp*-musllinux*"
CIBW_TEST_COMMAND: >
Expand All @@ -75,12 +67,8 @@ jobs:
CIBW_ARCHS_LINUX: aarch64
CIBW_BUILD_FRONTEND: build
CIBW_ENVIRONMENT_LINUX : >
TA_LIBRARY_PATH="ta-lib-install/lib"
TA_INCLUDE_PATH="ta-lib-install/include"
PIP_NO_BUILD_ISOLATION=false
CIBW_REPAIR_WHEEL_COMMAND_LINUX: |
export LD_LIBRARY_PATH="/project/ta-lib-install/lib:$LD_LIBRARY_PATH"
auditwheel repair -w {dest_dir} {wheel}
TA_BUNDLE_CACHE=/tmp/ta-lib-cache
- name: Upload wheels
uses: actions/upload-artifact@v7
with:
Expand All @@ -99,9 +87,6 @@ jobs:
uses: ilammy/msvc-dev-cmd@v1
with:
arch: amd64
- name: Build TA-Lib C Library
run: tools/build_talib_windows.cmd
shell: cmd
- name: Build wheels
uses: pypa/cibuildwheel@v4.2.1
env:
Expand All @@ -112,8 +97,6 @@ jobs:
MSBUILDTREATHIGHERTOOLSVERSIONASCURRENT: 1
CIBW_ARCHS_WINDOWS: AMD64
CIBW_ENVIRONMENT_WINDOWS: >
TA_LIBRARY_PATH="ta-lib-$TALIB_C_VER\\_build;$LIB"
TA_INCLUDE_PATH="ta-lib-$TALIB_C_VER\\include;$INCLUDE"
PIP_NO_BUILD_ISOLATION=false
- name: Upload wheel
uses: actions/upload-artifact@v7
Expand All @@ -133,9 +116,6 @@ jobs:
uses: ilammy/msvc-dev-cmd@v1
with:
arch: amd64_x86
- name: Build TA-Lib C library
run: tools/build_talib_windows.cmd
shell: cmd
- name: Build wheels
uses: pypa/cibuildwheel@v4.2.1
env:
Expand All @@ -146,8 +126,6 @@ jobs:
MSBUILDTREATHIGHERTOOLSVERSIONASCURRENT: 1
CIBW_ARCHS_WINDOWS: x86
CIBW_ENVIRONMENT_WINDOWS: >
TA_LIBRARY_PATH="ta-lib-$TALIB_C_VER\\_build;$LIB"
TA_INCLUDE_PATH="ta-lib-$TALIB_C_VER\\include;$INCLUDE"
PIP_NO_BUILD_ISOLATION=false
- name: Upload wheel
uses: actions/upload-artifact@v7
Expand All @@ -167,9 +145,6 @@ jobs:
uses: ilammy/msvc-dev-cmd@v1
with:
arch: arm64
- name: Build TA-Lib C Library
run: tools/build_talib_windows.cmd
shell: cmd
- name: Build wheels
uses: pypa/cibuildwheel@v4.2.1
env:
Expand All @@ -180,8 +155,6 @@ jobs:
MSBUILDTREATHIGHERTOOLSVERSIONASCURRENT: 1
CIBW_ARCHS_WINDOWS: ARM64
CIBW_ENVIRONMENT_WINDOWS: >
TA_LIBRARY_PATH="ta-lib-$TALIB_C_VER\\_build;$LIB"
TA_INCLUDE_PATH="ta-lib-$TALIB_C_VER\\include;$INCLUDE"
PIP_NO_BUILD_ISOLATION=false
- name: Upload wheel
uses: actions/upload-artifact@v7
Expand All @@ -200,20 +173,13 @@ jobs:
- name: Build wheels
uses: pypa/cibuildwheel@v4.2.1
env:
CIBW_BEFORE_ALL: |
bash ./tools/build_talib_macos.sh
CIBW_TEST_COMMAND: >
cd .. &&
pytest -k "not RSI and not threading" {project}/tests
CIBW_ARCHS_MACOS: x86_64
CIBW_ENVIRONMENT_MACOS : >
TA_LIBRARY_PATH="ta-lib-install/lib"
TA_INCLUDE_PATH="ta-lib-install/include"
PIP_NO_BUILD_ISOLATION=false
CIBW_REPAIR_WHEEL_COMMAND_MACOS: |
export MACOSX_DEPLOYMENT_TARGET=13.7
export DYLD_LIBRARY_PATH="/Users/runner/work/${{ github.event.repository.name }}/${{ github.event.repository.name }}/ta-lib-install/lib:$DYLD_LIBRARY_PATH"
delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}
TA_BUNDLE_CACHE=/tmp/ta-lib-cache
- name: Upload wheels
uses: actions/upload-artifact@v7
with:
Expand All @@ -231,20 +197,13 @@ jobs:
- name: Build wheels
uses: pypa/cibuildwheel@v4.2.1
env:
CIBW_BEFORE_ALL: |
bash ./tools/build_talib_macos.sh
CIBW_TEST_COMMAND: >
cd .. &&
pytest -k "not RSI and not threading" {project}/tests
CIBW_ARCHS_MACOS: arm64
CIBW_ENVIRONMENT_MACOS : >
TA_LIBRARY_PATH="ta-lib-install/lib"
TA_INCLUDE_PATH="ta-lib-install/include"
PIP_NO_BUILD_ISOLATION=false
CIBW_REPAIR_WHEEL_COMMAND_MACOS: |
export MACOSX_DEPLOYMENT_TARGET=14.0
export DYLD_LIBRARY_PATH="/Users/runner/work/${{ github.event.repository.name }}/${{ github.event.repository.name }}/ta-lib-install/lib:$DYLD_LIBRARY_PATH"
delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}
TA_BUNDLE_CACHE=/tmp/ta-lib-cache
- name: Upload wheels
uses: actions/upload-artifact@v7
with:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,6 @@ dist/
*~
*.egg-info/

talib/_ta_lib.c

!.github/
51 changes: 45 additions & 6 deletions DEVELOPMENT
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,24 @@ $ make build
$ [sudo] make install
```

TA-Lib C does not need to be installed first. `make build` uses an installed
TA-Lib only if its version is at least `TALIB_C_VER` (see `setup.py`); otherwise
it builds a static copy from the release tarball and links it in. Set
`TA_INCLUDE_PATH` / `TA_LIBRARY_PATH` to point at your own build instead.


Here's the full list of make commands (see the Makefile file):

make build # builds and places libs in the project directory; required for testing
make clean # cleans the local build files
make install # installs talib system-wide
make generate: # generates a fresh _func.pxi, _stream.pxi file. Requires talib and TA-Lib to both be installed
make perf # run performance profiling
make test # run tests
make build # builds and places libs in the project directory; required for testing
make build-bundled # same, but always builds and links the bundled static TA-Lib C
make clean # cleans the local build files
make install # installs talib system-wide
make generate: # generates a fresh _func.pxi, _stream.pxi file. Requires talib and TA-Lib to both be installed
make perf # run performance profiling
make test # run tests
make build-tsan # clean, then rebuild with ThreadSanitizer (needs a TSAN CPython)
make test-tsan # build-tsan, then run the threading tests under TSAN


The source code is comprised of one python package, located in the talib
directory, which itself has one Cython module (_ta_lib) which consists of
Expand Down Expand Up @@ -50,3 +60,32 @@ tools/generate_func.py,generate_stream.py

If you are interested in developing new indicator functions or whatnot on
the underlying TA-Lib, you must install TA-Lib from git.


Thread-safety testing with ThreadSanitizer (TSAN)

TSAN only works with a free-threaded CPython that was itself built with it
(Linux or macOS; clang is what CPython uses). Build one once, without
--with-pydebug so that NumPy wheels still install:

```
$ env CC=clang CXX=clang++ ./configure --disable-gil --with-thread-sanitizer \
--prefix=$HOME/py-tsan
$ make -j && make install
```

Then, from a virtual environment created with that interpreter (the extension
and TA-Lib C are both built with the compiler that interpreter was built with):

```
$ pip install -r requirements.txt pytest pandas
$ export TSAN_OPTIONS="suppressions=<cpython>/Tools/tsan/suppressions_free_threading.txt halt_on_error=1 handle_segv=0"
$ make test-tsan
```

TA_TSAN=1 (set by make build-tsan) builds the extension with -fsanitize=thread
and always builds a TSAN-instrumented static TA-Lib C alongside it (cached
separately). It is not supported on Windows. CI runs the full test suite this
way for 3.14t and 3.15t (see .github/workflows/tsan.yml).


Loading
Loading