CHANGELOG#
This file contains the list of changes made to the JLS project.
0.19.2#
2026 Oct 25 [in progress]
Updated documentation for doc.joulescope.com deployment.
0.19.1#
2026 Oct 16
Added Python 3.15 build.
0.19.0#
2026 Oct 13
BREAKING CHANGES
pyjls Writer.fsr_omit_data(signal_id, enable) was inverted and is now fixed. Previously,
enable=Truedisabled data omission andenable=Falseenabled it, the opposite of the documented behavior and of the C API. Nowenable=Trueomits constant data chunks andenable=Falsewrites all data chunks. If your code passedFalseto enable omission, change it toTrue. Omission only affects data types of 8 bits or less (u1, u4, u8, i4, i8), and readers reconstruct omitted data from the summaries, so existing files read the same. The C API, jls_wr_fsr_omit_data() and jls_twr_fsr_omit_data(), is unchanged.pyjls raises ValueError, not RuntimeError, for invalid arguments. When the JLS library rejects an argument with PARAMETER_INVALID, NOT_FOUND, NOT_SUPPORTED, TOO_BIG, or ALREADY_EXISTS, Writer and Reader methods now raise ValueError. Examples: an undefined signal_id, a non-FSR signal where FSR is required, reading past the end of a signal, or defining a signal twice. Other library errors, such as IO and MESSAGE_INTEGRITY, still raise RuntimeError. Code that catches RuntimeError for these cases must also catch ValueError. The C API now also returns PARAMETER_INVALID for negative lengths in jls_rd_fsr(), jls_rd_fsr_f32(), and jls_rd_fsr_statistics(), which previously succeeded without reading, and for an out-of-range jls_rd_tmap_get() index, which previously returned UNAVAILABLE.
Writers latch the first write error. After any write failure, such as a full disk, every later jls_wr_* and jls_twr_* call returns that error, and jls_wr_close() and jls_twr_close() return it instead of 0. Close then leaves the file not properly closed, without an END chunk, so jls_rd_open() recovers the data written before the error. The threaded writer reports errors from its thread on the next call, flush, or close instead of only logging them. pyjls Writer.close() raises RuntimeError when the file may be incomplete.
pyjls callback exceptions propagate. An exception raised in a Reader annotations(), user_data(), or utc() callback, or in a copy() or repair() callback, now propagates from the method, including KeyboardInterrupt. Previously it was logged and iteration silently stopped. close() called from a callback or from another thread while a method is running now raises RuntimeError instead of crashing.
Stricter signal and user data validation. jls_wr_signal_def() rejects definitions above the documented size caps (format.h) instead of crashing, and jls_wr_user_data() rejects chunk_meta values that use the reserved bits 15:12 instead of silently clearing them. Annotation and UTC decimate factors below 10 are raised to 10, and factors above 65536 are rejected. Readers skip invalid or duplicate signal definitions, and FSR definitions whose chunks exceed 16 MiB of sample data or 2**19 summary entries, or whose decimate factors do not divide as the writer has always aligned them (see format.h). jls_copy() keeps such signals by re-chunking them with the writer defaults.
Fixed-point signals are scaled. Statistics for data types with a nonzero fixed-point q now apply the documented 2**-q scale. Previously the scale was never applied. Summary chunks keep storing raw values, as in every earlier version, and the reader scales them.
Fixed the FSR statistics std computed from summaries. Summaries store the population std, but the reader treated it as the sample std, so windows spanning several summary entries reported a std that was too low by up to (k - 1) / k in variance, where k is the entry size (0.4% for k = 256). Statistics now report the sample std (n - 1) from both samples and summaries, so values change slightly. The file format is unchanged. See doc/plans/completed/summary_std_divisor.md.
Documented that a gap in FSR sample_ids is filled with NaN for float types and 0 otherwise, not interpolated.
Corrected the jls_rd_fsr_statistics() and Reader.fsr_statistics() documentation for a single window (data_length 1), which claimed sample-accurate results. Only windows shorter than 25 * sample_decimate_factor samples are computed from the samples. Longer windows combine summaries: min and max are exact, and mean and std are accurate to the summary precision (float32 summaries for types other than f64, u32, u64, i32, and i64). Behavior is unchanged.
Fixed the inverted Writer.fsr_omit_data() enable argument (see above). Corrected the jls_twr_fsr_omit_data() documentation, which said these data types are omitted regardless of the setting.
Fixed a crash on calling Writer.close() or Reader.close() twice, or on using a Writer or Reader after close(). close() freed the native handle but kept it, so the interpreter crashed with a segmentation fault. A second close() now does nothing, and the other methods raise ValueError. Closing a Reader inside its
withblock crashed the same way.Garbage collection now closes a Writer or Reader left open. Previously, the native handle leaked, and a Writer never finalized its file.
Documented thread safety. Writer methods may be called from multiple threads, except that close() must not overlap another call. Reader is not thread-safe; use one Reader per thread. Reader callbacks must not call the same Reader.
Fixed memory safety issues found by an argument validation review:
Writer.signal_def() with signal_id 256 or more wrote past the end of the Writer, in both the binding and jls_twr_signal_def().
Writer.fsr() and jls_twr_fsr() read signal state past the end of their tables for signal_id 256 or more. jls_twr_fsr() also read uninitialized state for undefined signals, and its byte count could overflow.
jls_core_signal_validate() returned success for undefined signals, so reading statistics, UTC, or time maps for one crashed.
Writing FSR data or fsr_omit_data to a VSR signal dereferenced NULL.
Reader.fsr() computed its buffer size in 32 bits, which wrapped for reads of 2**30 or more samples and overflowed the heap.
jls_rd_tmap_length() returned error codes as lengths, so Reader.time_map_get() returned uninitialized memory.
Fixed out-of-bounds reads in the threaded writer message log names, the time map interpolation search, and annotations whose data_size exceeds their chunk.
The threaded writer now rejects STRING and JSON payloads that are not NUL-terminated.
Reading FSR samples or statistics from a file whose level-1 index or summary timestamp is corrupt (with valid CRCs) read before the chunk buffer. These reads now return MESSAGE_INTEGRITY. Found by the read_fuzzer bitflip campaign under AddressSanitizer.
Fixed memory safety and logic defects found by a full design review (see doc/plans/completed/design_review_2026_10.md). Defects reachable from valid files or normal API use:
Reading u1 and u4 samples could write past the caller’s buffer, returned wrong samples, and failed some valid reads across chunks.
Writing u1 and u4 data lost the final partial byte, over-read the caller’s buffer by one byte, and stored wrong samples for duplicate sample_ids. An odd u4 overlap looped forever, growing the file.
Chunks over 1 MiB, such as large user data, could not be read, and chunks of about 1 MiB hung the reader.
Filling a sample_id gap for data types wider than 8 bits wrote heap memory into the file.
A write error followed by more writes overflowed the index and summary buffers. The threaded writer could hang on close.
Threaded writer messages near the queue size overflowed its buffer.
Omitted constant I4 and I8 data chunks could not be read.
Summary statistics for I32, U32, I64, U64, and F64 signals always failed: writers stored only half of each 64-bit summary. New files store full summaries. Existing files compute exact statistics from the samples. Sample statistics now support 64-bit data types.
I24 and U24 statistics used uninitialized memory.
Statistics with a huge increment overflowed the heap.
Nested reads inside annotation and user data callbacks silently truncated the outer iteration.
The reader leaked its time map for each FSR signal.
pyjls Writer could deadlock when the library logged from its thread, and an open Writer could hang interpreter exit.
Hardened reading of corrupt or hostile files with valid CRCs: chunk lengths that wrap or exceed the file, signal definitions that divide by zero or overflow, untrusted entry counts and timestamps in data, index, and summary chunks, recovery that overflowed or appended junk to a failed file, jls_copy() over-reads, jls_repair() head rewrites, and a chunk scan that skipped half the file or looped forever. Corrupt file errors return MESSAGE_INTEGRITY, so pyjls raises RuntimeError, not ValueError, for them.
Fixed regressions found by a second review of this release (doc/plans/completed/regression_review_2026_10_12.md): a writer closed after a latched error wrote END, which hid the samples after the last flushed summary from the reader; jls_copy() dropped a signal whose definition exceeded the new writer caps; jls_statistics_combine() discarded the min and max of a window holding both +inf and -inf; statistics computed from samples allocated the whole window, which the legacy 64-bit summary fallback made as large as the signal, and ran up to 40% slower than 0.18; pyjls Reader.time_map_length() raised for VSR signals instead of returning 0.
Fixed
pyjls csvon files with signals of different time extents, such as several Joulescopes recorded together. It used the latest end time instead of the earliest, so the shorter signals were read past their end and the export failed with PARAMETER_INVALID.pyjls extractclamps each signal to its own length when converting the common time window to sample ids, andpyjls plothandles signals shorter than the requested point count.jls_wr_flush() treats an fsync() that the file system does not support (EINVAL, ENOTSUP) as success instead of latching JLS_ERROR_IO, so an export to such a mount no longer reports an incomplete file.
Statistics skip NaN samples, and windows with no finite samples return NaN. UTC time maps ignore entries whose timestamps do not increase, so after a backward clock step, entries are skipped until UTC passes the last accepted entry.
pyjls accepts os.PathLike paths, raises ValueError for out-of-range integer arguments, returns int64 sample_ids from time maps, and releases the GIL while opening, copying, and repairing files. Library log messages, including those from the writer thread, reach the pyjls logger on the next pyjls call rather than immediately, and are counted and dropped beyond 256 pending messages. This prevents a deadlock.
On Windows, closing a threaded writer waits for its thread to finish, as on other platforms, instead of freeing it after 1 second.
Errors that the JLS library returns for invalid arguments (PARAMETER_INVALID, NOT_FOUND, NOT_SUPPORTED, TOO_BIG, ALREADY_EXISTS) now raise ValueError instead of RuntimeError. The library validates signal_id and annotation_type, and the threaded writer now reports these errors synchronously, rather than only logging them from its thread.
All C API functions now validate their arguments: NULL instances, output pointers, and data with a nonzero length return PARAMETER_INVALID instead of crashing, as do negative lengths for jls_rd_fsr(), jls_rd_fsr_f32(), and jls_rd_fsr_statistics(), which previously succeeded without reading. jls_rd_tmap_get() returns PARAMETER_INVALID, not UNAVAILABLE, for an index out of range. jls_twr_fsr_f32() rejects non-f32 signals, which read past the caller’s buffer. STRING and JSON user_data and annotations require data. The statistics functions, jls_crc32c(), and jls_log_printf() ignore NULL.
Fixed memory leaks when jls_wr_open() or jls_copy() fail to open a file.
Empty data now works: Writer.annotation() with data None or b’’, Writer.fsr() with an empty array, and Reader.fsr() or Reader.fsr_statistics() with length 0.
An exception in a Reader.annotations(), user_data(), or utc() callback now stops the iteration and is logged. Previously, user_data() and utc() continued.
Writer.annotation() now stores a non-finite y as NAN, as intended.
Added example/recovery_fuzz.py, which damages pyjls-written files and reads them back through the pyjls Reader, for use with an AddressSanitizer build.
Added the JLS_COVERAGE CMake option and a Linux CI job that publishes a gcovr C unit test coverage report. See doc/coverage.md.
Fixed a double free in jls_rd_open() when crash recovery failed to close the file.
Fixed the reader after a transient read or seek error. A failed seek could permanently drop a signal’s summary levels, so later jls_rd_fsr_length() calls returned 0. A failed read at the end of the UTC track could cache an incomplete time map for jls_rd_sample_id_to_timestamp() and jls_rd_timestamp_to_sample_id(). A failed chunk header read inside the file now returns JLS_ERROR_IO, not JLS_ERROR_EMPTY.
Added test-only read and seek fault injection, jls_bk_read_limit_set(), and test/read_fault_test.c, which sweeps an I/O failure through the reader, crash recovery, jls_repair(), and jls_copy(). ctest runs a sampled sweep; run
read_fault_test --fullfor every fault point (about a minute). writer_test also runs fewer cases by default, with the same coverage, and every case with--full.Fixed crash recovery and jls_repair() after an I/O error. A failed read was treated as damage or as the end of the data, so recovery could truncate or drop valid data, or leave a file that no longer opened. Recovery and repair now stop with JLS_ERROR_IO and leave the file as after a crash at that point. jls_raw_chunk_seek() now returns JLS_ERROR_MESSAGE_INTEGRITY, not JLS_ERROR_IO, for a corrupt offset.
Fixed jls_copy() after a read error. It skipped the chunk as damaged and reported success with an incomplete copy. It now fails with JLS_ERROR_IO. Opening a file now also fails if a seek fails while finding the file end. See doc/plans/completed/recovery_io_faults.md.
0.18.0#
2026 Aug 9
Dropped python 3.11 support and updated build dependencies to latest.
Reworked recovery of damaged JLS files (doc/plans/completed/recovery_defects.md findings F1-F9 and doc/plans/completed/recovery_single_pass.md):
jls_rd_open now recovers tail truncation, the common damage from writer crashes, power loss, and arbitrary truncation, quickly and robustly in O(crash tail) regardless of file size:
Fixed recovery truncating a fully valid final chunk (backward scan miss).
Fixed recovery overwriting valid chunks below the truncation point; all repair writes append at the file end.
Relinked the orphan final chunk, restored lost track head offsets, and clamped dangling chain links in the crash window.
Removed the full-file traversals from open: track head search considers only applicable track types, and the data chain clamp starts from the last valid level-1 index entry. For signals with long omitted (constant) tails, the clamp start search interleaves with a data chain walk from the head so neither a long omitted tail nor a long data prefix forces an O(file) walk.
jls_rd_open returns JLS_ERROR_TRUNCATED when the file needs truncation recovery but is read-only or the recovery write fails (was JLS_ERROR_IO).
jls_rd_open returns JLS_ERROR_MESSAGE_INTEGRITY on any other corruption (torn track heads, interior bit errors or zeroed regions, damaged definition chains) instead of opening degraded; every chain and index hop validates chunk tag and meta, which also fixes heap overflow and infinite read loops on damaged files. API impact: callers that tolerated degraded opens must catch this and offer jls_repair.
Added jls_repair(): offline, in-place repair with a single full-file traversal for the damage classes that jls_rd_open refuses. Relinks chains around dead chunks, rebuilds torn or lost track heads, converts dead FSR data to omitted regions reconstructed from summaries, and finalizes the file. Includes the
jls repairCLI command and the pyjlsrepair()binding.Repaired annotation, UTC, and user data chains; UTC iteration now returns entries written after the last summary.
Fixed reader and raw instance leaks on failed open.
Fixed writer out-of-bounds read on byte-aligned FSR writes.
Added jls_msg_fn and jls_progress_fn (jls/progress.h); the jls_copy callback types are deprecated aliases. Copy and repair callbacks now honor the cancel return value and stop with JLS_ERROR_ABORTED; pyjls callbacks receive str messages and cancel by returning True.
Added jls_raw_rd_header_total() and jls_raw_rd_payload_total() read accounting; the recovery test suite asserts open cost is independent of file size.
Added corruption fuzz campaigns to the read_fuzzer example (truncate, zero_tail, zero_hole, torn_update, bitflip, write_prefix), which verify recovery and jls_repair end to end, plus the crash-recovery and offline-repair unit test suites.
0.17.0#
2026 Apr 27
Added windows-11-arm64 support.
0.16.0#
2025 Dec 2
Dropped python 3.10 support and added python 3.14 support.
Bumped packaging versions.
Attempted win11-arm64 support, but GitHub runners do not support C++.
0.15.0#
2025 Jun 13
Added “csv” entry point.
0.14.1#
2025 May 21
Improved timestamp_patch entry point.
Allow integer timestamp
Read/write in chunks
0.14.0#
2025 May 6
Fixed Reader to use 2018-01-01T00:00:00.0000000Z (time64 0) when UTC not found #15
Added ISO 8601 support to time64.
Added timestamp_patch entry point.
0.13.3#
2025 Apr 22
Improved “copy” to properly close output file on error.
Modified read to check header magic before checksum.
0.13.2#
2025 Apr 10
Fixed FSR read when no summaries exist.
Fixed FSR to write remaining potentially omitted 1 & 4 bit data when closing file.
Fixed jls_rd_fsr for 1 bit and 4 bit data when sample_id is unaligned.
0.13.1#
2025 Apr 9
Fixed TimeMap.time_map_get in Python binding. Expanded unit test coverage.
0.13.0#
2025 Mar 29
Improved Reader time_map API.
Added jls_rd_tmap_length and jls_rd_tmap_get
Added jls_tmap_length and jls_tmap_get
Added Python Reader methods time_map_length and time_map_get
Added Iterable support to Python Reader sample_id_to_timestamp and timestamp_to_sample_id.
Improved unit test coverage.
0.12.1#
2025 Mar 27
Added Linux ARM64 builds.
0.11.1#
2024 Dec 5
Added “group_id” documentation to pyjls.Writer.annotate.
Fixed annotate entry point.
0.11.0#
2024 Oct 29
Migrated to numpy 2.
Added Python 3.13 support.
Dropped Python 3.9 support.
0.10.0#
2024 Aug 21
Improved Python binding to simplify finding sources and signals.
Added pyjls.source_lookup.
Improved pyjls.signal_lookup to support ‘<source_name>.<signal_name>’ format.
Added “–timestamp” option to export to add relative time to CSV output.
0.9.7#
2024 Jul 25
Fixed plot entry point to time align signals.
0.9.6#
2024 Jun 28
Fixed issue with stop time in extract entry point.
Pinned numpy < 2 for build environment.
0.9.5#
2024 May 29
Fixed u4 data type reconstruction when omitted for compression #13
0.9.4#
2024 May 10
Fixed unicode filename support on Windows #12
0.9.3#
2024 Mar 7
Added “export” python subcommand.
0.9.2#
2024 Feb 14
Fixed index error in file repair operation.
Fixed FSR summary reconstruction on truncation #10
0.9.1#
2023 Nov 13
Bumped rev to fix publish to pypi.
0.9.0#
2023 Nov 13
Added jls_copy with “jls copy” command and pyjls “copy” entry point.
Publish release assets including “jls.exe”.
0.8.2#
2023 Oct 25
Added file truncation repair for never closed files.
Added support for python 3.12.
0.8.1#
2023 Sep 18
Improved python binding Reader.fsr_statistics documentation #9.
Fixed occasional missing last byte in jls_rd_fsr for 1-bit signals.
0.8.0#
2023 Sep 16
Added file truncation repair.
Added jls_raw_truncate and jls_raw_backend.
Refactored reader and writer into new core to enable repairer.
Added jls_twr_flags_get/set.
Added JLS_TWR_FLAG_DROP_ON_OVERFLOW which drops samples on overflow.
Added jls_wr_fsr_omit_data and jls_twr_fsr_omit_data.
Automatically omit constant 1, 4, & 8 bit entry FSR data chunks.
Modified default signal_def settings for improved performance.
Added jls_dt_str for pretty-printing the data type.
Added u1 and u4 FSR data type support to performance tool.
Fixed jls_rd_fsr out of bounds memory access.
Added read_fuzzer tool to jls example.
0.7.3#
2023 Jul 24
Added “noexcept” to python callbacks. Cython 3.0 deprecates implicit noexcept.
0.7.2#
2023 Jun 14
Fixed build for Raspberry Pi.
Added JLS_OPTIMIZE_CRC cmake option.
0.7.1#
2023 Jun 7
Added Reader.signal_lookup.
Improved documentation.
Added Read The Docs integration.
Improved python “export” subcommand to specify “–signal” by name.
Added python subcommand “plot”.
Improved build process, migrated to GitHub Actions.
Bumped minimum python version from 3.8 to 3.9.
0.7.0#
2023 May 31
Fixed incorrect write timestamp stride in FSR index/summary entries. Any recording over 5.77 hours was incorrect.
Improved threaded writer.
Removed jls_wr_flush during close due to UI performance problems.
Release the GIL on some python Writer operations.
Reduced buffer size from 100,000,000 B to 64 MB.
Save string null termination byte for annotations and user_data.
Increased thread priority on Windows.
Do not quit until all messages are processed.
Added jls executable to examples.
Improved reader logging and error handling.
0.6.3#
2023 May 16
Added support for building a shared library. Initialize build subdir with “cmake -DBUILD_SHARED_LIBS=ON ..”
0.6.2#
2023 Apr 28
Improved UTC read processing.
Added “–utc” option to info entry point.
Fixed incorrect NaNs in summary on write.
Added “export” pyjls entry point.
0.6.1#
2023 Apr 27
Added FSR support for missing and duplicate data.
Added FSR support for unaligned u1 and u4 data.
Improved log messages.
0.6.0#
2023 Apr 26
Fixed JLS to handle non-zero sample_id for first FSR data sample.
Added pyjls.Reader.timestamp_to_sample_id and sample_id_to_timestamp.
0.5.3#
2023 Apr 19
Fixed build warnings for fn() declarations.
0.5.2#
2023 Mar 30
Reduced default log level to WARNING.
0.5.1#
2023 Mar 16
Added zero length check to jls_wr_fsr_data.
0.5.0#
2023 Mar 9
Added support for data_type strings (not just enum integers).
Modified python reader bindings to release GIL.
Migrated to time64 representation for all API calls. Use utc_to_jls and jls_to_utc to convert as needed to/from
python timestamps.Added data_type_as_enum and data_type_as_str conversion functions.
Improved exception messages.
0.4.3#
2022 Nov 30
Changed windows dependency from deprecated pypiwin32 to pywin32.
Bumped numpy dependencies.
Added build system requirements for pip.
0.4.2#
2022 Mar 17
Fixed pyjls build for Raspberry Pi OS 64-bit.
0.4.1#
2022 Mar 7
Fixed build for Linux and macOS.
0.4.0#
2022 Mar 5
Added support for additional data types (was only f32):
Signed Integer: i4, i8, i16, i24, i32, i64
Unsigned Integer: u1, u4, u8, u16, u24, u32, u64
Float: f64
Fixed point signed & unsigned integers.
Renamed jls_rd_fsr_f32_statistics to jls_rd_fsr_statistics, which now always returns statistics as double (f64).
Improved documentation.
0.3.4#
2021 Oct 28
Added all version fields to pyjls module.
Fixed writer not correctly serializing null and empty strings #6.
0.3.3#
2021 Jul 7
Added reader sample_id bounds checks to FSR functions.
Cached jls_rd_fsr_length results.
0.3.2#
2021 Jul 7
Fixed incorrect statistics computation when using summaries.
Added example/jls_read.c.
Connected example/generate.py arguments to work correctly.
Fixed documentation for jls_rd_fsr_f32_statistics().
Added GitHub action.
0.3.1#
2021 Apr 13
Fixed y annotation argument order.
Added horizontal marker annotation.
Fixed Python API to automatically convert to/from UTC python timestamps.
Fixed reader seek when contains multiple annotations at same timestamp.
0.3.0#
2021 Apr 8
Yanked this release, use 0.3.1
NOT BACKWARDS COMPATIBLE with 0.2.6 and earlier.
Modified annotation to contain optional y-axis position. API change.
Improved file format consistency and improved format.h.
Added UTC track writer and reader for FSR signals.
Added INDEX and SUMMARY writer for ANNOTATION and UTC tracks.
Added reader seek to timestamp for ANNOTATION and UTC using INDEX & SUMMARY.
0.2.6#
2021 Mar 18
Fixed uninitialized variables in POSIX backend (thanks Valgrind!).
Fixed memory leaks (thanks Valgrind!).
0.2.5#
2021 Mar 18
Added 32-bit ARMv7 support (Raspberry Pi OS).
Added 64-bit ARM support for Apple silicon (M1).
0.2.4#
2021 Mar 16
Added support for aarch64 (Raspberry Pi 4). Untested on mac M1.
Fixed POSIX time.
0.2.3#
2021 Mar 10
Fixed sdist to include native files for pyjls cython build.
Added CREDITS.html.
Included credits, license & readme with pyjls source distribution.
0.2.2#
2021 Mar 9
Fixed build using Clang and AppleClang.
0.2.1#
2021 Mar 8
Fixed timeout on close when bursting data.
Added annotate_stress example.
Added flush.
0.2.0#
2021 Mar 8
Added group_id parameter to annotation.
Fixed example/generate.py length argument.
Updated numpy dependency to 1.20.
Fixed pyjls console_script.
Changed logging interface and connected native to python logging.
0.1.0#
2021 Mar 1
Initial public release.
0.0.1#
2021 Feb 5
Initial documentation.