Skip to content

The shiboken6 bindings

How _pyside6_scintilla is built, what shiboken generates, problems encountered while wrapping ScintillaEditBase (Phase 1), and how to update everything when a new Scintilla release is vendored.

How it fits together

  • src/scintilla/ -- vendored Scintilla release tarball, unmodified (see .gitattributes; see docs/auditing.md for how to verify this).
  • src/scintilla_qt/CMakeLists.txt -- builds the vendored Scintilla core plus its Qt widget glue (PlatQt, ScintillaQt, ScintillaEditBase) as a shared library, scintilla_qt.
  • src/pyside6_scintilla/bindings/
  • bindings.h -- header shiboken's ApiExtractor parses to discover the C++ declarations referenced by bindings.xml.
  • bindings.xml -- the shiboken typesystem: declares which C++ types/enums/classes get Python wrappers.
  • CMakeLists.txt -- runs shiboken to generate the wrapper sources below, then compiles and links them (with scintilla_qt and Qt) into _pyside6_scintilla.{pyd,so,dylib}.
  • src/pyside6_scintilla/__init__.py -- re-exports the compiled extension's public API as the pyside6_scintilla package.

Generated files

shiboken6 generates these files (under build/<tag>/src/pyside6_scintilla/bindings/_pyside6_scintilla/) from bindings.h + bindings.xml. They are build artifacts, not checked in, and are not hand-edited or hand-commented.

File Purpose Generated by
_pyside6_scintilla_module_wrapper.cpp Module init (PyInit__pyside6_scintilla); registers all wrapped types with the extension module shiboken6, from the <typesystem> root
scintilla_wrapper.cpp / .h Wrapper for the Scintilla namespace and its enums (Update, Message, KeyMod, ...) shiboken6, from <namespace-type name="Scintilla">
scintilla_notificationdata_wrapper.cpp / .h Wrapper for the Scintilla::NotificationData value type shiboken6, from <value-type name="NotificationData">
scintillaeditbase_wrapper.cpp / .h Wrapper for the ScintillaEditBase widget's public methods/signals/slots shiboken6, from <object-type name="ScintillaEditBase"/>
_pyside6_scintilla_python.h Master header included by every wrapper .cpp; includes bindings.h plus the PySide6/shiboken headers it needs shiboken6

The GENERATED_SOURCES list in bindings/CMakeLists.txt only needs the .cpp files -- the .h files are pulled in by those .cpp files.

_pyside6_scintilla_python.h is the key one to remember: it has its own include list, separate from bindings.h. Anything done in bindings.h (e.g. #define/#undef tricks) only affects what shiboken's ApiExtractor sees while generating code -- it has no effect on what the generated .cpp files compile against. Fixes that need to apply at compile time belong in bindings/CMakeLists.txt (e.g. as target_compile_definitions).

Issues encountered (Phase 1: ScintillaEditBase)

1. Duplicate Scintilla::Message enum codegen

ScintillaStructures.h forward-declares enum class Message; ("in case ScintillaMessages.h isn't included") in addition to the full definition in ScintillaMessages.h. Both are in namespace Scintilla, and <enum-type name="Message"/> matches both. shiboken's clang-based parser treats them as two separate EnumDecls and emits duplicate converters/type-index entries for Scintilla::Message (SBK_Scintilla_Message_IDX defined twice, etc.) -- this fails to compile. Reordering includes does not help.

Fix, in bindings.h: include ScintillaMessages.h first (the real Message definition), then wrap #include <ScintillaStructures.h> in #define Message ScintillaMessageFwdDeclUnused / #undef Message so its forward declaration becomes an unrelated, unused enum instead of a second Scintilla::Message. As a side effect, NotificationData::message (whose type is now that renamed dummy enum) has to be dropped via <modify-field name="message" remove="true"/> on NotificationData in bindings.xml.

A leftover, mostly cosmetic warning remains: "Scoped enum Scintilla::ScintillaMessageFwdDeclUnused does not have a type entry" -- consistent with several other pre-existing warnings for unbound scoped enums and safe to ignore.

2. FindText/FindTextW macro collision (Windows)

<windows.h> is pulled in transitively via Python.h before any Qt header gets a chance to define WIN32_LEAN_AND_MEAN. That drags in <commdlg.h>, which #defines FindText to FindTextA/FindTextW. This mangles Scintilla::Message::FindText in the generated wrapper .cpp files -- and since those files don't include bindings.h (see "Generated files" above), an #undef FindText there has no effect on the compile step.

Fix, in bindings/CMakeLists.txt (Windows only): target_compile_definitions(_pyside6_scintilla PRIVATE WIN32_LEAN_AND_MEAN). This stops <commdlg.h> from ever being pulled in, so FindText is never redefined.

3. Qt6Core5Compat.dll not bundled by the PySide6 wheel (Windows)

scintilla_qt links Qt6::Core5Compat (vendored Scintilla's PlatQt.cpp uses QTextCodec, removed from QtCore in Qt6). Qt6Core5Compat.dll is the one Qt6*.dll the PySide6 wheel does not bundle, so _pyside6_scintilla.pyd -> scintilla_qt.dll -> Qt6Core5Compat.dll fails to load at import time.

Fix, in bindings/CMakeLists.txt (Windows only): install(FILES $<TARGET_FILE:Qt6::Core5Compat> DESTINATION pyside6_scintilla) ships Qt6Core5Compat.dll alongside scintilla_qt.dll/_pyside6_scintilla.pyd, where Windows' default same-directory DLL search finds it. Also requires scintilla_qt itself to be installed (RUNTIME/LIBRARY DESTINATION pyside6_scintilla), and src/pyside6_scintilla/__init__.py to import PySide6.QtWidgets before importing the compiled extension, so PySide6 registers its own DLL search directory for Qt6Widgets.dll/Qt6Core.dll/pyside6.abi3.dll/ shiboken6.abi3.dll first.

Not yet addressed for Linux/macOS wheels: Qt6Core5Compat bundling there will need delvewheel/auditwheel/delocate in CI, or an install(FILES ...) step analogous to the Windows one. _pyside6_scintilla's INSTALL_RPATH is already set to $ORIGIN (Linux)/@loader_path (macOS) in anticipation of this.

4. ScintillaEditBase/ScintillaEdit signals carrying Scintilla::Position/enum-typed parameters never reach a Python slot

shiboken marshals Scintilla::Position/ModificationFlags/FoldLevel/etc. fine for ordinary method arguments (they're registered as <primitive-type>/<enum-type> in bindings.xml), but Qt's generic meta-call path -- used to invoke a Python slot from a C++ signal emission -- can't convert them. Connecting a Python slot to e.g. ScintillaEditBase.modified looked like it worked (connect() raised nothing), but the slot was never actually called: Qt's event loop swallows the resulting TypeError. Registering these as Qt meta-types was considered and rejected in favor of re-emission (below), to avoid baking Scintilla-specific Q_DECLARE_METATYPE registration into a generic binding layer.

Fix: bindings/scintilla_signal_fixes.h (a hand-written, non-vendored header, included from bindings.h) declares two subclasses, ScintillaEditBaseFixed/ScintillaEditFixed, that connect each affected signal in C++ to a lambda re-emitting the same data as a plain-typed (int/QByteArray/QString) signal of the same name -- the same pattern already used by ScintillaDocument::modified (see src/scintilla/qt/ScintillaEdit/ScintillaDocument.h). PySide creates a Signal descriptor for each signal it finds in a class's own QMetaObject, so the subclass's redeclaration simply shadows the inherited broken-signature one through normal Python attribute/MRO lookup -- the same mechanism that lets a Python subclass override a method by just redefining it, no typesystem changes involved. src/pyside6_scintilla/__init__.py imports the Fixed classes aliased to the public ScintillaEditBase/ScintillaEdit names, so this is invisible from the public API.

A few non-obvious wrinkles along the way:

  • generate="no" breaks header include order. The first attempt set generate="no" on ScintillaEditBase/ScintillaEdit (since ScintillaEditBaseFixed/ScintillaEditFixed are the only Python-visible classes) but that drops the header from shiboken's "Bound library includes" list in the generated module header, which is sorted alphabetically -- losing ScintillaEditBase.h (which internally includes ScintillaTypes.h before ScintillaStructures.h) left ScintillaStructures.h included before ScintillaTypes.h with nothing to satisfy its need for Position first. Fix: keep ScintillaEditBase/ScintillaEdit fully bound (default generate="yes") -- a <modify-function remove="all"/> on the broken signal overloads was tried too, on the assumption it'd avoid an ambiguous Python overload, but turned out to be a no-op (see the next point) and was dropped as dead weight.
  • <modify-function> doesn't apply to Qt signals. Unlike ordinary methods, Qt signals are exposed to Python via PySide's own metaobject introspection at class-creation time, not via shiboken's typesystem-driven method generation -- so a <modify-function signature="..." remove="all"/> targeting a signal silently has no effect (confirmed: removing the dozen remove="all" rules entirely, the build and full test suite behaved identically). The fix above never needed them -- the shadowing happens purely through normal Python class attribute/MRO lookup, the same mechanism a Python subclass uses to override any other inherited attribute by just redefining it.
  • moc can't parse the vendored headers. ScintillaEditBaseFixed/ ScintillaEditFixed need Q_OBJECT, so CMake's AUTOMOC runs moc over scintilla_signal_fixes.h -- but moc's lightweight parser chokes on ScintillaStructures.h's more intricate C++ once it's transitively included. moc only actually needs the base class names for the generated metaobject, not their full definitions. Fix: guard the real includes with #ifdef Q_MOC_RUN / #else, giving moc bare forward declarations of ScintillaEditBase/ScintillaEdit instead.
  • EXPORT_IMPORT_API only makes sense across a DLL boundary. Vendored ScintillaEditBase/ScintillaEdit use this macro because they live in scintilla_qt.dll and are imported by _pyside6_scintilla. Applying the same macro to ScintillaEditBaseFixed/ScintillaEditFixed -- which are defined and compiled entirely within _pyside6_scintilla, no DLL boundary crossed -- resolved to __declspec(dllimport) on Windows and collided with moc's own definition of staticMetaObject (error C2491: definition of dllimport static data member not allowed). Fix: don't apply EXPORT_IMPORT_API to classes fully contained within one compiled module.
  • <modify-function> only applies to the declaring class. shiboken ignores a <modify-function> rule registered against a subclass that merely inherits the method rather than redeclaring it. get_doc()'s <define-ownership owner="target"/> rule (giving Python ownership of the returned ScintillaDocument) has to stay on <object-type name="ScintillaEdit">, where get_doc() is actually declared -- registering it on ScintillaEditFixed (a subclass that just inherits it) silently no-ops, and the document's signals kept firing even after its Python wrapper was dropped.

See tests/test_scintilla_edit_base.py::test_modified_signal_reaches_python_slot for the regression test.

Type stubs (_pyside6_scintilla.pyi)

src/pyside6_scintilla/_pyside6_scintilla.pyi (plus the PEP 561 src/pyside6_scintilla/py.typed marker) gives Pylance/mypy/ruff/pylint full signatures and autocomplete for Scintilla.*/ScintillaEditBase. Unlike the generated wrapper .cpp/.h files above, it's checked into git -- it directly improves the local dev/IDE experience (e.g. for examples/simple_scintilla_base_edit/main.py) and is picked up by wheel.packages for published wheels, without adding a PySide6-importing step to the CMake/CI build.

Regenerate it with make stubs (tools/generate_pyi.py) after rebuilding the extension (make install or uv sync --reinstall-package pyside6-scintilla), and whenever bindings.xml/bindings.h change the public API surface. The script:

  1. Runs shiboken6-genpyi against the built _pyside6_scintilla extension, working around two issues with the generic shiboken6-genpyi entry point on a PySide6-based binding (both detailed in the script's docstring):
  2. it re-imports the extension from a bare path, which fails with "DLL load failed" unless pyside6_scintilla (and therefore PySide6.QtWidgets, see issue 3 above) has been imported first;
  3. its find_imports() references a PySide6 global that only PySide6.support.generate_pyi sets up -- the generic entry point raises NameError: name 'PySide6' is not defined without it.
  4. Aliases Scintilla.Position/sptr_t/uptr_t to int. These are primitive typedefs (<primitive-type> in bindings.xml), not nested types, but genpyi emits unresolvable Scintilla.Position-style forward references for them (ScintillaEditBase.event_command's signature, NotificationData's # type: comments). If a future Scintilla release adds another such primitive typedef, genpyi will warn (RuntimeWarning: ... UNRECOGNIZED: 'Scintilla.<name>') -- add it to PRIMITIVE_ALIASES in tools/generate_pyi.py.
  5. Stitches the # ... doc comments from src/scintilla/include/Scintilla.iface into Scintilla.Message enum members as docstrings (per-member hover docs). val/enum-style features generally don't have these comments, so this mainly benefits Message.

_pyside6_scintilla.pyi is excluded from ruff (extend-exclude in pyproject.toml) since it's machine-generated and not ruff-format-compliant.

Lifetime & ownership

Audit of the wrapped types for memory-safety/lifetime surprises beyond standard PySide6/Qt semantics (Phase 3 Tracer 6).

ScintillaEditBase/ScintillaEdit as QWidgets

No caveat: standard Qt/PySide6 ownership rules apply. A parentless widget is kept alive by Python references, and is destroyed -- its signals included -- once the last reference is dropped. A widget with a Qt parent is kept alive by that parent regardless of Python references, and its signals (notify, command, macroRecord, etc.) keep firing as long as the widget exists.

NotificationData (delivered via notify)

ScintillaEditBase.notify(NotificationData *pscn) hands Python a wrapper around a transient struct that Scintilla reuses across notifications.

  • Reading a field during the handler call is safe -- e.g. text (a const char * in C++) is eagerly copied into an independent Python str at access time, valid for as long as you like afterwards.
  • The NotificationData object itself must not be retained past the handler call: once later notifications overwrite the shared struct, its fields no longer reflect the notification it was originally delivered for (see tests/test_scintilla_notify.py).

Where available, prefer ScintillaEditBase's typed per-notification signals (modified, charAdded, marginClicked, etc.) instead -- e.g. modified's text parameter is already a QByteArray, with no such caveat.

ScintillaDocument from get_doc()

Already documented on ScintillaEdit.get_doc()'s docstring and its reference page: the returned ScintillaDocument has no Qt parent, so it's kept alive only by your Python reference to it -- drop it and its modified/ save_point/etc. signals stop firing, even though the underlying document buffer stays alive via Scintilla's own refcounting as long as an editor uses it (see tests/test_scintilla_document.py).

Updating to a new Scintilla release

Per docs/specs/mission.md, Scintilla updates are deliberate and tested, not automatic. When vendoring a new release tarball:

  1. Replace src/scintilla/ with the new release's contents (remove the old tree first so deleted upstream files are actually removed, then extract the new tarball with --strip-components=1). .gitattributes (src/scintilla/** -text) keeps the diff clean.
  2. Diff src/scintilla/qt/ScintillaEditBase/ScintillaEditBase.pro against the previous release's copy. Update the source list in src/scintilla_qt/CMakeLists.txt to match any added/removed/renamed .cxx/.h files.
  3. Check src/scintilla/include/ScintillaTypes.h, ScintillaMessages.h, and ScintillaStructures.h for new enums, typedefs, or struct fields that ScintillaEditBase's public API (or ScintillaEdit, for Phase 2) now references. Add the corresponding <primitive-type>, <enum-type>, or <value-type> entries to bindings.xml -- this is iterative: run the build, read the shiboken error naming the missing type, add the minimal entry, repeat.
  4. Re-check whether the duplicate-Message-enum workaround (issue 1 above) still applies as written. If Scintilla restructures its headers, the #define <Type> ... / #include / #undef pattern in bindings.h may need to move, target a different type, or no longer be needed -- the same pattern applies if a different enum/struct gets a forward-declaration collision in a future release.
  5. Bump __version__ in src/pyside6_scintilla/__init__.py from X.Y.Z.N to the new Scintilla release's X.Y.Z.0 (binding-only changes against the same Scintilla release increment N).
  6. Rebuild with uv sync --reinstall-package pyside6-scintilla (uv sync alone doesn't always pick up CMake/typesystem changes), then uv run pytest and uv run ruff check .. See docs/build.md.