11 Commits
Author SHA1 Message Date
woggioni c79f268471 Add OIDC back-channel logout with session indexes 2026-10-28 07:50:14 +00:00
woggioni 88024c49ca Migrate dependency management from pip to uv
- Add a uv workspace at the repository root covering all packages/kaya-* members
- Replace pip-compile requirements files with a committed uv.lock
- Remove redundant per-package dev extras
- Build and publish artifacts with uv build and uv publish in CI
- Update the README with uv sync/run/build commands
2026-10-21 01:02:08 +00:00
woggioni ad6e97e20a Add reusable component schemas to kaya-openapi
Operations could only inline schemas: the generated document had no
components section, so #/components/... references dangled. OpenAPIMixin
and generate_spec now accept a components mapping (merged verbatim into
the document) and kaya.openapi exports schema_ref() to build
#/components/schemas/<name> references. Component references are
validated at generation time and unknown ones raise ValueError.
2026-10-14 11:52:03 +00:00
woggioni e73c4b2c18 Validate ID tokens against discovered issuer and refreshed tokens
CI / Build Pip package (push) Successful in 8m49s
OIDCClient.validate_id_token compared the iss claim against
OIDCConfig.issuer (the discovery URL) instead of the issuer advertised by
the provider metadata. This broke Microsoft Entra ID, whose v2 metadata
advertises the templated issuer
https://login.microsoftonline.com/{tenantid}/v2.0 while tokens carry the
concrete tenant GUID, and it also affected tenant-domain issuers.

- Validate iss against the discovered metadata issuer and substitute
  {tenantid} with the token's tid claim.
- Catch jwt.PyJWTError in the OIDC callback so invalid tokens return 400
  instead of 500 (InvalidIssuerError is not a ValueError).
- Re-enable aud/exp/iat/nbf checks for unsigned (alg: none) ID tokens.
- Validate ID tokens returned by the refresh grant before storing them,
  including nonce continuity when present and sub continuity with the
  session user.
- Document Entra ID support and update tests.
2026-10-01 23:22:00 +08:00
woggioni fa8bb0487d Add Pushed Authorization Request (RFC 9126) support
CI / Build Pip package (push) Successful in 14m8s
OIDCConfig.par selects never (default), auto, or always. When enabled, build_authorization_url pushes the authorization parameters to the provider's PAR endpoint and redirects the browser with only client_id and request_uri. Providers that require PAR raise a clear error when configured with par='never'. Client authentication follows the token endpoint (HTTP Basic or public client).
2026-10-01 08:23:40 +08:00
woggioni 15dc4b6dad Add single/multi identity strategy to OIDC providers
OIDCProviders and OIDCMixin accept strategy='single-identity' (default) or 'multi-identity'. Single-identity evicts other providers' identities and tokens after a successful login so the session holds exactly one user, matching Spring Security's last-login-wins behavior. Ephemeral flow keys are preserved so in-flight logins can still complete.
2026-10-01 08:23:40 +08:00
woggioni 3af2e7a975 Add support for multiple OIDC providers
Namespace per-provider session state as oidc:{provider}:... so several providers can share one session, and add an OIDCProviders mixin that wires multiple configs onto distinct /auth/{provider}/... routes. Logging out of one provider no longer signs the user out of the others.
2026-10-01 08:23:40 +08:00
woggioni 540a7b111e Add standard attributes to kaya-otel websocket spans 2026-09-30 08:58:06 +08:00
woggioni 7c46badebb Surface vanished websocket peers as WebSocketDisconnect
CI / Build Pip package (push) Successful in 4m55s
Granian's RSGI transport raises RSGIProtocolClosed when the peer is gone
without a close handshake, which escaped receive() and was logged as an
application error, and sends to the dead transport raised the raw Granian
exception as well.

Treat a closed RSGI transport like the ASGI websocket.disconnect message:
receive() reports a close message so async-for loops end normally, while
writes (accept/send/close) raise a new kaya.core.WebSocketDisconnect
carrying close code 1006. KayaApp.handle_websocket swallows that exception
as normal termination, so after hooks still run and disconnects are no
longer reported as errors. The ASGI adapter maps its server's OSError
disconnect (e.g. uvicorn's ClientDisconnected) to the same exception for
consistent semantics across backends.
2026-09-29 22:10:16 +08:00
woggioni 148a35b71c Add kaya-otel package for OpenTelemetry tracing and metrics
CI / Build Pip package (push) Successful in 4m8s
Instrument HTTP requests and WebSocket connections via Kaya hooks, covering both ASGI and RSGI. Records handler exceptions, WebSocket close codes, optional header capture, exclusions and lifecycle hooks.

Add route-template resolution and exception visibility to kaya-core so trace/metric attributes can use low-cardinality routes and failed spans can record escaped exceptions.
2026-09-19 00:39:15 +00:00
woggioni 69762d93df Document that mixin setup/shutdown loops may not be running
Under RSGI __rsgi_init__ is called before the loop starts, so
asyncio.get_running_loop() raises RuntimeError inside setup(). Spell out
that mixins must use the loop they are given, since the ASGI lifespan
incidentally runs with a live loop.
2026-09-18 02:37:08 +00:00
62 changed files with 6758 additions and 499 deletions

No files matched your search

+65 -51
View File
@@ -5,7 +5,7 @@ on:
- 'release/*' - 'release/*'
jobs: jobs:
build_pip_package: build_pip_package:
name: "Build Pip package" name: "Build and publish packages"
runs-on: hostinger runs-on: hostinger
timeout-minutes: 30 timeout-minutes: 30
container: container:
@@ -18,12 +18,14 @@ jobs:
fetch-tags: true fetch-tags: true
- uses: actions/setup-python@v6 - uses: actions/setup-python@v6
with: with:
python-version: '3.14.6' python-version-file: '.python-version'
cache: 'pip' - name: Install uv
uses: astral-sh/setup-uv@v10
with:
version: '0.13.4'
enable-cache: true
- name: Install dependencies - name: Install dependencies
run: | run: uv sync --locked
python -m venv .venv
.venv/bin/pip install -r requirements-dev.txt
- name: Check kaya-core - name: Check kaya-core
run: | run: |
.venv/bin/python -m mypy -p kaya.core .venv/bin/python -m mypy -p kaya.core
@@ -60,75 +62,87 @@ jobs:
run: | run: |
.venv/bin/python -m mypy -p kaya.forwarded .venv/bin/python -m mypy -p kaya.forwarded
.venv/bin/python -m unittest discover -s packages/kaya-forwarded/tests .venv/bin/python -m unittest discover -s packages/kaya-forwarded/tests
- name: Check kaya-otel
run: |
.venv/bin/python -m mypy -p kaya.otel
.venv/bin/python -m unittest discover -s packages/kaya-otel/tests
- name: Publish kaya-core artifacts - name: Publish kaya-core artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-core uv build --package kaya-core --out-dir packages/kaya-core/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-core/dist/*.whl packages/kaya-core/dist/*.tar.gz uv publish packages/kaya-core/dist/*.whl packages/kaya-core/dist/*.tar.gz
- name: Publish kaya-rsgi artifacts - name: Publish kaya-rsgi artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-rsgi uv build --package kaya-rsgi --out-dir packages/kaya-rsgi/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-rsgi/dist/*.whl packages/kaya-rsgi/dist/*.tar.gz uv publish packages/kaya-rsgi/dist/*.whl packages/kaya-rsgi/dist/*.tar.gz
- name: Publish kaya-session artifacts - name: Publish kaya-session artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-session uv build --package kaya-session --out-dir packages/kaya-session/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-session/dist/*.whl packages/kaya-session/dist/*.tar.gz uv publish packages/kaya-session/dist/*.whl packages/kaya-session/dist/*.tar.gz
- name: Publish kaya-session-memcache artifacts - name: Publish kaya-session-memcache artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-session-memcache uv build --package kaya-session-memcache --out-dir packages/kaya-session-memcache/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-session-memcache/dist/*.whl packages/kaya-session-memcache/dist/*.tar.gz uv publish packages/kaya-session-memcache/dist/*.whl packages/kaya-session-memcache/dist/*.tar.gz
- name: Publish kaya-session-redis artifacts - name: Publish kaya-session-redis artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-session-redis uv build --package kaya-session-redis --out-dir packages/kaya-session-redis/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-session-redis/dist/*.whl packages/kaya-session-redis/dist/*.tar.gz uv publish packages/kaya-session-redis/dist/*.whl packages/kaya-session-redis/dist/*.tar.gz
- name: Publish kaya-oidc artifacts - name: Publish kaya-oidc artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-oidc uv build --package kaya-oidc --out-dir packages/kaya-oidc/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-oidc/dist/*.whl packages/kaya-oidc/dist/*.tar.gz uv publish packages/kaya-oidc/dist/*.whl packages/kaya-oidc/dist/*.tar.gz
- name: Publish kaya-openapi artifacts - name: Publish kaya-openapi artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-openapi uv build --package kaya-openapi --out-dir packages/kaya-openapi/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-openapi/dist/*.whl packages/kaya-openapi/dist/*.tar.gz uv publish packages/kaya-openapi/dist/*.whl packages/kaya-openapi/dist/*.tar.gz
- name: Publish kaya-cors artifacts - name: Publish kaya-cors artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-cors uv build --package kaya-cors --out-dir packages/kaya-cors/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-cors/dist/*.whl packages/kaya-cors/dist/*.tar.gz uv publish packages/kaya-cors/dist/*.whl packages/kaya-cors/dist/*.tar.gz
- name: Publish kaya-forwarded artifacts - name: Publish kaya-forwarded artifacts
env: env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }} UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }} UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }} UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: | run: |
.venv/bin/pyproject-build packages/kaya-forwarded uv build --package kaya-forwarded --out-dir packages/kaya-forwarded/dist --no-sources
.venv/bin/twine upload --repository gitea packages/kaya-forwarded/dist/*.whl packages/kaya-forwarded/dist/*.tar.gz uv publish packages/kaya-forwarded/dist/*.whl packages/kaya-forwarded/dist/*.tar.gz
- name: Publish kaya-otel artifacts
env:
UV_PUBLISH_URL: ${{ vars.PYPI_REGISTRY_URL }}
UV_PUBLISH_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
UV_PUBLISH_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: |
uv build --package kaya-otel --out-dir packages/kaya-otel/dist --no-sources
uv publish packages/kaya-otel/dist/*.whl packages/kaya-otel/dist/*.tar.gz
+1
View File
@@ -0,0 +1 @@
3.14.6
+36 -36
View File
@@ -15,67 +15,67 @@ This repository is a monorepo for the Kaya framework. The code is split into ind
- **kaya-openapi** — automatic OpenAPI specification generation (`packages/kaya-openapi/`) - **kaya-openapi** — automatic OpenAPI specification generation (`packages/kaya-openapi/`)
- **kaya-cors** — CORS (Cross-Origin Resource Sharing) support (`packages/kaya-cors/`) - **kaya-cors** — CORS (Cross-Origin Resource Sharing) support (`packages/kaya-cors/`)
- **kaya-forwarded** — trusted-proxy `Forwarded`/`X-Forwarded-*` client address resolution (`packages/kaya-forwarded/`) - **kaya-forwarded** — trusted-proxy `Forwarded`/`X-Forwarded-*` client address resolution (`packages/kaya-forwarded/`)
- **kaya-otel** — OpenTelemetry tracing and metrics (`packages/kaya-otel/`)
Additional `kaya-*` packages can be added as new directories under `packages/`. Additional `kaya-*` packages can be added as new directories under `packages/`. When adding a new package, also add it to the root `pyproject.toml` dependencies and `[tool.uv.sources]` so `uv sync` installs it.
## Build & run locally ## Build & run locally
Install dev dependencies: This project uses [uv](https://docs.astral.sh/uv/) for dependency management. From the repository root:
```bash ```bash
pip install --index-url https://gitea.woggioni.net/api/packages/woggioni/pypi/simple --extra-index-url https://pypi.org/simple -r requirements-dev.txt uv sync
``` ```
Install the packages in development mode: This creates `.venv`, installs all workspace packages in editable mode, and installs the dev dependency group from `uv.lock`. Use `uv lock` after changing dependencies, and commit the updated `uv.lock`.
```bash
pip install -e packages/kaya-core -e packages/kaya-rsgi -e packages/kaya-session -e packages/kaya-session-redis -e packages/kaya-session-memcache -e packages/kaya-oidc -e packages/kaya-openapi -e packages/kaya-cors -e packages/kaya-forwarded
```
Run the example: Run the example:
```bash ```bash
python example/hello.py uv run python example/hello.py
``` ```
## Tests ## Tests
```bash ```bash
python -m unittest discover -s packages/kaya-core/tests uv run python -m unittest discover -s packages/kaya-core/tests
python -m unittest discover -s packages/kaya-rsgi/tests uv run python -m unittest discover -s packages/kaya-rsgi/tests
python -m unittest discover -s packages/kaya-session/tests uv run python -m unittest discover -s packages/kaya-session/tests
python -m unittest discover -s packages/kaya-session-redis/tests uv run python -m unittest discover -s packages/kaya-session-redis/tests
python -m unittest discover -s packages/kaya-session-memcache/tests uv run python -m unittest discover -s packages/kaya-session-memcache/tests
python -m unittest discover -s packages/kaya-oidc/tests uv run python -m unittest discover -s packages/kaya-oidc/tests
python -m unittest discover -s packages/kaya-openapi/tests uv run python -m unittest discover -s packages/kaya-openapi/tests
python -m unittest discover -s packages/kaya-cors/tests uv run python -m unittest discover -s packages/kaya-cors/tests
python -m unittest discover -s packages/kaya-forwarded/tests uv run python -m unittest discover -s packages/kaya-forwarded/tests
uv run python -m unittest discover -s packages/kaya-otel/tests
``` ```
## Static analysis ## Static analysis
```bash ```bash
mypy -p kaya.core uv run mypy -p kaya.core
mypy -p kaya.rsgi uv run mypy -p kaya.rsgi
mypy -p kaya.session uv run mypy -p kaya.session
mypy -p kaya.session.redis uv run mypy -p kaya.session.redis
mypy -p kaya.session.memcache uv run mypy -p kaya.session.memcache
mypy -p kaya.oidc uv run mypy -p kaya.oidc
mypy -p kaya.openapi uv run mypy -p kaya.openapi
mypy -p kaya.cors uv run mypy -p kaya.cors
mypy -p kaya.forwarded uv run mypy -p kaya.forwarded
uv run mypy -p kaya.otel
``` ```
## Building packages ## Building packages
```bash ```bash
python -m build packages/kaya-core uv build --package kaya-core --out-dir packages/kaya-core/dist
python -m build packages/kaya-rsgi uv build --package kaya-rsgi --out-dir packages/kaya-rsgi/dist
python -m build packages/kaya-session uv build --package kaya-session --out-dir packages/kaya-session/dist
python -m build packages/kaya-session-redis uv build --package kaya-session-redis --out-dir packages/kaya-session-redis/dist
python -m build packages/kaya-session-memcache uv build --package kaya-session-memcache --out-dir packages/kaya-session-memcache/dist
python -m build packages/kaya-oidc uv build --package kaya-oidc --out-dir packages/kaya-oidc/dist
python -m build packages/kaya-openapi uv build --package kaya-openapi --out-dir packages/kaya-openapi/dist
python -m build packages/kaya-cors uv build --package kaya-cors --out-dir packages/kaya-cors/dist
python -m build packages/kaya-forwarded uv build --package kaya-forwarded --out-dir packages/kaya-forwarded/dist
uv build --package kaya-otel --out-dir packages/kaya-otel/dist
``` ```
-5
View File
@@ -26,11 +26,6 @@ dependencies = [
"typing-extensions", "typing-extensions",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
+2 -1
View File
@@ -4,7 +4,7 @@ from ._http_context import HttpContext
from ._mixin import KayaMixin from ._mixin import KayaMixin
from ._tree import Tree, PathIterator from ._tree import Tree, PathIterator
from ._path_handler import PathHandler, Matches from ._path_handler import PathHandler, Matches
from ._websocket import WebSocket, WebSocketMessage from ._websocket import WebSocket, WebSocketDisconnect, WebSocketMessage
__all__ = [ __all__ = [
@@ -18,5 +18,6 @@ __all__ = [
'Matches', 'Matches',
'PathIterator', 'PathIterator',
'WebSocket', 'WebSocket',
'WebSocketDisconnect',
'WebSocketMessage' 'WebSocketMessage'
] ]
+17 -1
View File
@@ -9,7 +9,7 @@ from ._http_method import HttpMethod
from ._mixin import KayaMixin from ._mixin import KayaMixin
from ._path_handler import Context from ._path_handler import Context
from ._types import StrOrStrings from ._types import StrOrStrings
from ._websocket import WebSocket, WebSocketMessage from ._websocket import WebSocket, WebSocketDisconnect, WebSocketMessage
from ._asgi import AsgiContext, AsgiWebSocket from ._asgi import AsgiContext, AsgiWebSocket
from ._tree import Tree from ._tree import Tree
from ._types.asgi import LifespanScope, HTTPScope as ASGIHTTPScope, WebSocketScope as ASGIWebSocketScope from ._types.asgi import LifespanScope, HTTPScope as ASGIHTTPScope, WebSocketScope as ASGIWebSocketScope
@@ -145,6 +145,9 @@ class KayaApp(AbstractKayaApp):
await handler.handle_request(ctx, captured) await handler.handle_request(ctx, captured)
else: else:
await ctx.send_empty(404) await ctx.send_empty(404)
except Exception as exc:
ctx.exception = exc
raise
finally: finally:
for hook in reversed(self._after_request_hooks): for hook in reversed(self._after_request_hooks):
await hook(ctx) await hook(ctx)
@@ -161,6 +164,15 @@ class KayaApp(AbstractKayaApp):
await handler.handle_request(ws, captured) await handler.handle_request(ws, captured)
else: else:
await ws.close(1000) await ws.close(1000)
except WebSocketDisconnect:
# The peer went away while the handler was using the websocket
# (e.g. sending on a vanished transport). This is normal
# termination, not an application error, so it is not recorded
# on the socket nor re-raised.
pass
except Exception as exc:
ws.exception = exc
raise
finally: finally:
for hook in reversed(self._after_websocket_hooks): for hook in reversed(self._after_websocket_hooks):
await hook(ws) await hook(ws)
@@ -173,6 +185,10 @@ class KayaApp(AbstractKayaApp):
for mixin in self._mixins: for mixin in self._mixins:
mixin.shutdown(loop) mixin.shutdown(loop)
def route_template(self, url: str, method: HttpMethod = HttpMethod.GET) -> Optional[str]:
"""Return the registered route template that would handle ``url``."""
return self._tree.route_template(url, method)
def route(self, def route(self,
paths: StrOrStrings, paths: StrOrStrings,
methods: Optional[HttpMethod | Sequence[HttpMethod]] = None, methods: Optional[HttpMethod | Sequence[HttpMethod]] = None,
+14 -5
View File
@@ -17,7 +17,7 @@ from pwo import Maybe
from pathlib import Path from pathlib import Path
from ._http_method import HttpMethod from ._http_method import HttpMethod
from ._http_context import HttpContext from ._http_context import HttpContext
from ._websocket import WebSocket, WebSocketMessage from ._websocket import WebSocket, WebSocketDisconnect, WebSocketMessage
from ._types import StrOrStrings from ._types import StrOrStrings
from ._types.asgi import HTTPScope, WebSocketScope from ._types.asgi import HTTPScope, WebSocketScope
@@ -179,7 +179,16 @@ class AsgiWebSocket(WebSocket):
# Emit a list rather than a tuple: wsproto's AcceptConnection (used # Emit a list rather than a tuple: wsproto's AcceptConnection (used
# by httpx_ws and others) requires list concatenation. # by httpx_ws and others) requires list concatenation.
message['headers'] = list(encode_headers(headers)) message['headers'] = list(encode_headers(headers))
await self._send(message) await self._send_message(message)
async def _send_message(self, message: Mapping[str, Any]) -> None:
try:
await self._send(message)
except OSError as exc:
# ASGI servers (e.g. uvicorn) report a vanished peer with an
# OSError subclass; surface it as a kaya-level disconnect so
# handlers do not need to know about server internals.
raise WebSocketDisconnect(code=1006) from exc
async def receive(self) -> WebSocketMessage: async def receive(self) -> WebSocketMessage:
message = await self._receive() message = await self._receive()
@@ -200,13 +209,13 @@ class AsgiWebSocket(WebSocket):
return WebSocketMessage(kind='close') return WebSocketMessage(kind='close')
async def send_text(self, data: str) -> None: async def send_text(self, data: str) -> None:
await self._send({'type': 'websocket.send', 'text': data}) await self._send_message({'type': 'websocket.send', 'text': data})
async def send_bytes(self, data: bytes) -> None: async def send_bytes(self, data: bytes) -> None:
await self._send({'type': 'websocket.send', 'bytes': data}) await self._send_message({'type': 'websocket.send', 'bytes': data})
async def close(self, code: int = 1000) -> None: async def close(self, code: int = 1000) -> None:
await self._send({'type': 'websocket.close', 'code': code}) await self._send_message({'type': 'websocket.close', 'code': code})
async def __anext__(self) -> WebSocketMessage: async def __anext__(self) -> WebSocketMessage:
message = await self.receive() message = await self.receive()
@@ -29,6 +29,7 @@ class HttpContext(ABC):
server: Optional[Tuple[str, Optional[int]]] server: Optional[Tuple[str, Optional[int]]]
request_body: AsyncIterator[bytes] request_body: AsyncIterator[bytes]
session: Optional[Any] = None session: Optional[Any] = None
exception: Optional[BaseException] = None
@abstractmethod @abstractmethod
async def stream_body(self, async def stream_body(self,
+15 -2
View File
@@ -29,9 +29,22 @@ class KayaMixin(ABC):
pass pass
def setup(self, loop: AbstractEventLoop) -> None: def setup(self, loop: AbstractEventLoop) -> None:
"""Called on lifespan startup (default: no-op).""" """Called on lifespan startup (default: no-op).
Under RSGI (granian) the loop is NOT yet running when this is
called: schedule work on the passed ``loop`` (e.g.
``loop.create_task``) instead of calling
``asyncio.get_running_loop()``, which raises ``RuntimeError``
there. Under ASGI the lifespan runs inside a coroutine, so a
running loop happens to be available — do not rely on that.
"""
pass pass
def shutdown(self, loop: AbstractEventLoop) -> None: def shutdown(self, loop: AbstractEventLoop) -> None:
"""Called on lifespan shutdown (default: no-op).""" """Called on lifespan shutdown (default: no-op).
As with :meth:`setup`, use the passed ``loop``; it may already be
stopped under RSGI, so ``asyncio.get_running_loop()`` is not
reliable here either.
"""
pass pass
+31
View File
@@ -237,6 +237,37 @@ class Tree:
# return (handler, unmatched) # return (handler, unmatched)
return None return None
def route_template(self, url: str, method: HttpMethod = HttpMethod.GET) -> Optional[str]:
"""Return the registered route template that would handle ``url``.
Static segments are returned verbatim, ``${name}``/``${name:int}``
parameter matchers are reconstructed from the matched nodes, and glob
matchers keep their pattern. Returns ``None`` when no route (including
recursive fallback routes) would handle the request.
"""
result = self.find_node((p for p in PathIterator(urlparse(url).path)), method)
if result is None:
return None
node, captured = result
if len(captured.unmatched_paths) > 0 and not any(handler.recursive for handler in node.handlers):
return None
parts: List[str] = []
current: Optional[Node | PathMatcher] = node
while current is not None:
if isinstance(current, Node):
key = current.key
if not isinstance(key, HttpMethod) and key != '/':
parts.append(str(key))
elif isinstance(current, IntMatcher):
parts.append('${%s:int}' % current.name)
elif isinstance(current, StrMatcher):
parts.append('${%s}' % current.name)
elif isinstance(current, GlobMatcher):
parts.append(current.pattern)
current = current.parent
return '/' + '/'.join(reversed(parts)) if parts else '/'
def parse(self, leaf: str, parent: Optional[Node | PathMatcher]) -> Node | PathMatcher: def parse(self, leaf: str, parent: Optional[Node | PathMatcher]) -> Node | PathMatcher:
start = 0 start = 0
result = index_of_with_escape(leaf, '${', '\\', 0) result = index_of_with_escape(leaf, '${', '\\', 0)
@@ -13,6 +13,41 @@ class WebSocketMessage:
data: WebSocketData = None data: WebSocketData = None
class WebSocketDisconnect(Exception):
"""Raised when using a websocket whose peer has disconnected.
Concrete websocket implementations raise this from ``send_text()`` and
``send_bytes()`` when the transport reports that the peer is gone (for
example when a browser tab is closed without a close handshake), so
handlers can catch it and stop producing output:
.. code-block:: python
try:
await ws.send_text('hello')
except WebSocketDisconnect:
pass
Reading from a disconnected websocket is not an error: ``receive()``
reports a ``close`` message and iterating with ``async for`` simply ends,
which is what most handlers should rely on instead.
``code`` is a websocket close code; ``1006`` means the connection was
closed abnormally, without a close handshake.
"""
code: int
reason: str
def __init__(self, code: int = 1006, reason: str = '') -> None:
self.code = code
self.reason = reason
message = f'WebSocket disconnected (code={code})'
if reason:
message = f'{message}: {reason}'
super().__init__(message)
class WebSocket(ABC): class WebSocket(ABC):
path: str path: str
query_string: str query_string: str
@@ -21,6 +56,7 @@ class WebSocket(ABC):
client: Optional[Tuple[str, int]] client: Optional[Tuple[str, int]]
server: Optional[Tuple[str, Optional[int]]] server: Optional[Tuple[str, Optional[int]]]
session: Optional[Any] = None session: Optional[Any] = None
exception: Optional[BaseException] = None
@abstractmethod @abstractmethod
async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None: async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
+29
View File
@@ -250,3 +250,32 @@ class AsgiTest(unittest.TestCase):
r = await client.put("/restaurants/42/menu") r = await client.put("/restaurants/42/menu")
self.assertEqual(404, r.status_code) self.assertEqual(404, r.status_code)
def test_route_template(self):
self.assertEqual('/employee/${employee_id}',
self.app.route_template('/employee/101325', HttpMethod.GET))
self.assertEqual('/square/${x:int}', self.app.route_template('/square/30', HttpMethod.GET))
self.assertIsNone(self.app.route_template('/unknown', HttpMethod.GET))
@async_test
async def test_exception_exposed_to_after_hooks(self):
app = KayaApp()
seen = []
async def after_request(ctx: HttpContext) -> None:
seen.append(ctx.exception)
app.add_after_request_hook(after_request)
@app.GET('/raises')
async def raises(ctx: HttpContext) -> None:
raise RuntimeError('boom')
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as client:
with self.assertRaises(RuntimeError):
await client.get('/raises')
self.assertEqual(1, len(seen))
self.assertIsInstance(seen[0], RuntimeError)
self.assertEqual('boom', str(seen[0]))
+21
View File
@@ -80,6 +80,27 @@ class TreeTest(unittest.TestCase):
self.assertIs(Maybe.of(handler_num).map(self.handlers.__getitem__).or_none(), self.assertIs(Maybe.of(handler_num).map(self.handlers.__getitem__).or_none(),
Maybe.of_nullable(res).map(lambda it: it[0]).or_none()) Maybe.of_nullable(res).map(lambda it: it[0]).or_none())
def test_route_template(self):
cases: Tuple[Tuple[str, HttpMethod, Optional[str]], ...] = (
('/home/something', HttpMethod.GET, '/home/something'),
('/home/something_else', HttpMethod.POST, '/home/something_else'),
('/home/README.md', HttpMethod.GET, '/home/*.md'),
('/home/something/ciao/blah/README.md', HttpMethod.GET, '/home/something/*/blah/*.md'),
('/home/bar/ciao/blah/README.md', HttpMethod.GET, '/home/bar/*'),
('/unknown', HttpMethod.GET, None),
)
for url, method, expected in cases:
with self.subTest(f'{method} {url}'):
self.assertEqual(expected, self.tree.route_template(url, method))
def test_route_template_with_params(self):
tree = Tree()
tree.add((p for p in ('foo', '${id:int}')), HttpMethod.PUT, self.handlers[0])
tree.add((p for p in ('foo', '${name}')), HttpMethod.GET, self.handlers[1])
self.assertEqual('/foo/${id:int}', tree.route_template('/foo/42', HttpMethod.PUT))
self.assertEqual('/foo/${name}', tree.route_template('/foo/bar', HttpMethod.GET))
self.assertIsNone(tree.route_template('/foo/bar', HttpMethod.DELETE))
def test_two_method_agnostic_matchers_raise(self): def test_two_method_agnostic_matchers_raise(self):
tree = Tree() tree = Tree()
tree.add((p for p in ('foo', '*')), None, self.handlers[0]) tree.add((p for p in ('foo', '*')), None, self.handlers[0])
+113 -3
View File
@@ -1,9 +1,9 @@
import unittest import unittest
import httpx import httpx
from pwo import async_test from pwo import async_test
from httpx_ws import aconnect_ws, WebSocketDisconnect from httpx_ws import aconnect_ws, WebSocketDisconnect as HttpxWebSocketDisconnect
from httpx_ws.transport import ASGIWebSocketTransport from httpx_ws.transport import ASGIWebSocketTransport
from kaya.core import KayaApp, WebSocket from kaya.core import KayaApp, WebSocket, WebSocketDisconnect, WebSocketMessage
from kaya.core._asgi import AsgiWebSocket from kaya.core._asgi import AsgiWebSocket
@@ -66,7 +66,7 @@ class WebSocketTest(unittest.TestCase):
transport = ASGIWebSocketTransport(app=self.app) transport = ASGIWebSocketTransport(app=self.app)
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client: async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:
with self.assertRaises(WebSocketDisconnect) as cm: with self.assertRaises(HttpxWebSocketDisconnect) as cm:
async with aconnect_ws("/unknown", client) as ws: async with aconnect_ws("/unknown", client) as ws:
pass pass
@@ -151,3 +151,113 @@ class WebSocketTest(unittest.TestCase):
} }
ws = AsgiWebSocket(scope, receive, send) ws = AsgiWebSocket(scope, receive, send)
self.assertEqual('ws', ws.scheme) self.assertEqual('ws', ws.scheme)
class FakeWebSocket(WebSocket):
"""Minimal in-memory websocket for exercising ``KayaApp`` directly."""
def __init__(self, path: str) -> None:
self.path = path
self.query_string = ''
self.scheme = 'ws'
self.headers: dict[str, list[str]] = {}
self.client = ('127.0.0.1', 12345)
self.server = ('127.0.0.1', 80)
self.closed: list[int] = []
async def accept(self, headers=None) -> None:
pass
async def receive(self) -> WebSocketMessage:
return WebSocketMessage(kind='close')
async def send_text(self, data: str) -> None:
pass
async def send_bytes(self, data: bytes) -> None:
pass
async def close(self, code: int = 1000) -> None:
self.closed.append(code)
async def __anext__(self) -> WebSocketMessage:
raise StopAsyncIteration
class WebSocketDisconnectTest(unittest.TestCase):
def setUp(self):
self.app = KayaApp()
@self.app.websocket('/disconnect')
async def disconnect(ws: WebSocket) -> None:
await ws.accept()
await ws.send_text('before')
raise WebSocketDisconnect()
@self.app.websocket('/failure')
async def failure(ws: WebSocket) -> None:
await ws.accept()
raise RuntimeError('boom')
@async_test
async def test_disconnect_terminates_quietly(self):
after_hook_calls = []
async def after(ws: WebSocket) -> None:
after_hook_calls.append(ws)
self.app.add_after_websocket_hook(after)
ws = FakeWebSocket('/disconnect')
await self.app.handle_websocket(ws)
self.assertEqual([ws], after_hook_calls)
self.assertIsNone(ws.exception)
@async_test
async def test_other_errors_are_recorded_and_reraised(self):
ws = FakeWebSocket('/failure')
with self.assertRaises(RuntimeError):
await self.app.handle_websocket(ws)
self.assertIsInstance(ws.exception, RuntimeError)
@async_test
async def test_asgi_send_after_disconnect_raises_websocket_disconnect(self):
async def send(message):
raise OSError('client gone')
async def receive():
return {'type': 'websocket.disconnect', 'code': 1006}
ws = AsgiWebSocket(self._scope(), receive, send)
with self.assertRaises(WebSocketDisconnect) as ctx:
await ws.send_text('hello')
self.assertEqual(1006, ctx.exception.code)
with self.assertRaises(WebSocketDisconnect):
await ws.send_bytes(b'hello')
with self.assertRaises(WebSocketDisconnect):
await ws.close()
@async_test
async def test_asgi_accept_after_disconnect_raises_websocket_disconnect(self):
async def send(message):
raise OSError('client gone')
async def receive():
return {'type': 'websocket.connect'}
ws = AsgiWebSocket(self._scope(), receive, send)
with self.assertRaises(WebSocketDisconnect) as ctx:
await ws.accept()
self.assertEqual(1006, ctx.exception.code)
@staticmethod
def _scope() -> dict:
return {
'type': 'websocket',
'path': '/echo',
'query_string': b'',
'scheme': 'ws',
'client': ('127.0.0.1', 12345),
'server': ('127.0.0.1', 80),
'headers': [],
}
-5
View File
@@ -25,11 +25,6 @@ dependencies = [
"kaya-core", "kaya-core",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws", "kaya-rsgi"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
-5
View File
@@ -25,11 +25,6 @@ dependencies = [
"kaya-core", "kaya-core",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws", "kaya-rsgi"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
+199
View File
@@ -40,20 +40,219 @@ async def profile(ctx: HttpContext):
`OIDCMixin` depends on `SessionMixin`; passing only `oidc` to `KayaApp(mixins=...)` `OIDCMixin` depends on `SessionMixin`; passing only `oidc` to `KayaApp(mixins=...)`
also works because the app applies mixin dependencies automatically. also works because the app applies mixin dependencies automatically.
## Multiple providers
Use `OIDCProviders` to run several OIDC providers in one application. Each
config's `provider` becomes a route segment under `base_path`
(`/auth/{provider}/login`, `/auth/{provider}/callback` and
`/auth/{provider}/logout`), and all providers share a single `SessionMixin`:
```python
from kaya.oidc import OIDCConfig, OIDCProviders
session = SessionMixin(InMemorySessionStore())
oidc = OIDCProviders(
[
OIDCConfig(
issuer='https://accounts.google.com',
client_id=os.environ['GOOGLE_CLIENT_ID'],
client_secret=os.environ['GOOGLE_CLIENT_SECRET'],
redirect_uri='http://localhost:8000/auth/google/callback',
provider='google',
),
OIDCConfig(
issuer='https://github.com/login/oauth', # or any OIDC issuer
client_id=os.environ['GITHUB_CLIENT_ID'],
client_secret=os.environ['GITHUB_CLIENT_SECRET'],
redirect_uri='http://localhost:8000/auth/github/callback',
provider='github',
),
],
session=session,
base_path='/auth', # optional
)
app = KayaApp(mixins=[session, oidc])
@app.GET('/google-profile')
@oidc.require_auth('google')
async def google_profile(ctx: HttpContext):
user = oidc.get_user(ctx, 'google')
await ctx.send_str(200, f"Hello {user.email or user.sub}")
@app.GET('/who')
async def who(ctx: HttpContext):
users = oidc.get_users(ctx) # {provider: OIDCUser}
providers = oidc.logged_in_providers(ctx) # ['google', ...]
await ctx.send_str(200, ', '.join(users))
```
Each provider's session state is namespaced as `oidc:{provider}:...`. Make sure
each config's `redirect_uri` points at that provider's derived callback path.
### Identity strategy
`strategy` controls how many identities may coexist in the shared session:
- `'single-identity'` (default): completing a login evicts every other
provider's identity and tokens, so the session holds exactly one user — last
login wins, like Spring Security. After a GitHub login, `get_user(ctx,
'google')` returns `None` and `require_auth('google')` redirects to the Google
login.
- `'multi-identity'`: each provider stays logged in independently. Use it for
linked accounts, aggregating several provider APIs, or step-up flows. Logging
out of one provider keeps the shared session alive while another provider is
still logged in.
```python
oidc = OIDCProviders([google_config, github_config], session=session,
strategy='multi-identity')
```
Ephemeral flow keys (`state`, `nonce`, `code_verifier`) are never evicted, so
logins already in flight can still complete; under `'single-identity'` the last
callback to finish wins. Under either strategy, `get_users(ctx)` only reports
providers actually present in the session.
The composable alternative is to create one `OIDCMixin` per provider, pointing
each at distinct `login_path`/`callback_path`/`logout_path` values and sharing
the same `SessionMixin`. The same `strategy` keyword is accepted and defaults to
`'single-identity'`:
```python
google = OIDCMixin(google_config, session=session)
github = OIDCMixin(github_config, session=session, strategy='multi-identity')
app = KayaApp(mixins=[session, google, github])
```
> **Upgrading:** session keys are now prefixed with `oidc:{provider}:` (the
> default provider is `default`). Existing logged-in users are signed out once
> and simply re-authenticate.
## Back-channel logout
Set `backchannel_logout_path` to accept OIDC Back-Channel Logout tokens from
the provider:
```python
OIDCConfig(
issuer=os.environ['OIDC_ISSUER'],
client_id=os.environ['OIDC_CLIENT_ID'],
client_secret=os.environ['OIDC_CLIENT_SECRET'],
redirect_uri='https://example.com/auth/callback',
backchannel_logout_path='/auth/backchannel-logout',
)
```
Register the corresponding absolute URL (for example
`https://example.com/auth/backchannel-logout`) as the client's back-channel
logout URI with the provider. This registration happens at the provider; it is
not part of OIDC discovery. With `OIDCProviders`, enabling the option derives
`{base_path}/{provider}/backchannel-logout` for that provider.
The endpoint accepts form-encoded `POST` requests containing `logout_token`.
It validates the JWT signature against the provider JWKS, the `logout+jwt`
type, issuer, audience, `iat`, `jti`, and the back-channel logout event, and
rejects tokens containing `nonce`. A token with `sid` logs out sessions for
that provider session; without `sid`, `sub` logs out all sessions for that
subject. Under `multi-identity`, only the named provider's identity is removed
from a shared session. Valid requests return `200` with
`Cache-Control: no-store`, including when the session is already gone. Invalid
requests return `400`.
Back-channel logout requires a session store with index support. All built-in
stores (`InMemorySessionStore`, `RedisSessionStore`, and
`MemcacheSessionStore`) support it. `OIDCMixin` raises immediately if the
configured store does not. Sessions created before upgrading are indexed on
their next request.
Logout tokens must be asymmetrically signed; unsigned tokens, HMAC signatures,
and encrypted logout tokens (JWE) are not supported. `jti` is required, but no
replay cache is maintained: repeating a valid logout request is idempotent.
`backchannel_logout_leeway` controls the allowed clock skew (60 seconds by
default). For a templated Microsoft Entra issuer, the logout token must carry
`tid`, just as for ID token validation.
## Pushed authorization requests (PAR)
`OIDCConfig.par` controls RFC 9126 Pushed Authorization Requests:
- `'never'` (default): the authorization request travels as query parameters,
as before.
- `'auto'`: push the request whenever the provider advertises a
`pushed_authorization_request_endpoint`; otherwise fall back to the plain
redirect.
- `'always'`: always push; login raises a `RuntimeError` if the provider does
not support PAR.
```python
OIDCConfig(..., par='auto')
```
A provider that advertises `require_pushed_authorization_requests: true` must be
configured with `par='auto'` or `par='always'`. With `par='never'` the login
fails fast with a clear error instead of issuing a redirect the provider would
reject.
The same parameters as the plain authorization request (PKCE challenge,
`state`, `nonce`, scopes, ...) are POSTed directly to the provider, and the
browser is redirected with only `client_id` and the single-use `request_uri`.
Request parameters therefore no longer travel through the user agent. PKCE,
`state`, and `nonce` are still used and protect against request URI swapping.
Client authentication follows the token endpoint: HTTP Basic when
`client_secret` is set, or none for public clients. PAR with other client
authentication methods (`private_key_jwt`, mTLS) is not supported.
## Features ## Features
- Generic OIDC discovery - Generic OIDC discovery
- Authorization Code Flow with PKCE (S256) - Authorization Code Flow with PKCE (S256)
- ID token signature validation with JWKS - ID token signature validation with JWKS
- Issuer validation against discovery metadata, including support for
Microsoft Entra ID `{tenantid}` issuer templates
- Refreshed ID token validation on token refresh
- `state` and `nonce` protection - `state` and `nonce` protection
- Session fixation defense via `regenerate_id()` after login - Session fixation defense via `regenerate_id()` after login
- Optional userinfo endpoint fetch - Optional userinfo endpoint fetch
- Refresh token support - Refresh token support
- RP-initiated logout (when provider advertises `end_session_endpoint`) - RP-initiated logout (when provider advertises `end_session_endpoint`)
- Back-channel logout (`sid` and `sub` session invalidation)
- Pushed Authorization Requests (RFC 9126) via `OIDCConfig.par`
- Multiple providers in one app via `OIDCProviders` (or several `OIDCMixin` instances)
- `single-identity` (default) or `multi-identity` session strategy
- Composable with any other `KayaMixin` (RSGI, MCP, etc.) - Composable with any other `KayaMixin` (RSGI, MCP, etc.)
## Microsoft Entra ID
Entra ID v2 ID tokens carry an `iss` claim of the form
`https://login.microsoftonline.com/<tenant-id>/v2.0`, while the discovery
metadata for the multi-tenant endpoints advertises the templated issuer
`https://login.microsoftonline.com/{tenantid}/v2.0`. `kaya-oidc` validates the
`iss` claim against the discovered metadata and substitutes the token's `tid`
claim into `{tenantid}`, so both tenant-specific and multi-tenant issuers work:
```python
OIDCConfig(
issuer='https://login.microsoftonline.com/<tenant-id-or-domain>/v2.0',
client_id=os.environ['OIDC_CLIENT_ID'],
client_secret=os.environ['OIDC_CLIENT_SECRET'],
redirect_uri='https://example.com/auth/callback',
)
```
Use a tenant-specific issuer (the tenant GUID or a verified domain name) to
restrict logins to that tenant. The `common` and `organizations` aliases are
multi-tenant: any tenant in that cloud can obtain an ID token that passes
validation. If you build a multi-tenant application, check the `tid` claim
against the tenants you trust.
## Security notes ## Security notes
- The `iss` claim is validated against the `issuer` value of the provider's
discovery document, not against `OIDCConfig.issuer` (they can differ, for
example for Entra ID multi-tenant endpoints).
- Refreshed ID tokens are validated before being stored; the `nonce` is also
checked when the refreshed token carries one.
- The `none` signing algorithm is rejected by default. - The `none` signing algorithm is rejected by default.
- Only algorithms listed in `OIDCConfig.allowed_id_token_algorithms` are accepted. - Only algorithms listed in `OIDCConfig.allowed_id_token_algorithms` are accepted.
- Always use HTTPS for `redirect_uri` in production. - Always use HTTPS for `redirect_uri` in production.
-5
View File
@@ -28,11 +28,6 @@ dependencies = [
"PyJWT[crypto]", "PyJWT[crypto]",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
+6 -2
View File
@@ -1,11 +1,15 @@
from ._client import OIDCClient from ._client import OIDCClient
from ._config import OIDCConfig from ._config import OIDCConfig, PARPolicy
from ._mixin import OIDCMixin, OIDCUser from ._mixin import IdentityStrategy, OIDCMixin, OIDCUser
from ._providers import OIDCProviders
__all__ = [ __all__ = [
'IdentityStrategy',
'OIDCClient', 'OIDCClient',
'OIDCConfig', 'OIDCConfig',
'OIDCMixin', 'OIDCMixin',
'OIDCProviders',
'OIDCUser', 'OIDCUser',
'PARPolicy',
] ]
+186 -17
View File
@@ -1,3 +1,4 @@
import re
from typing import Any, Mapping, Optional, Sequence, cast from typing import Any, Mapping, Optional, Sequence, cast
from urllib.parse import urlencode from urllib.parse import urlencode
@@ -5,9 +6,13 @@ import httpx
import jwt import jwt
from jwt import PyJWK from jwt import PyJWK
from ._config import OIDCConfig from ._config import PARPolicy, OIDCConfig
from ._utils import generate_nonce, generate_pkce, generate_state from ._utils import generate_nonce, generate_pkce, generate_state
_TENANT_ID_PLACEHOLDER = re.compile(r'\{tenantid\}', re.IGNORECASE)
_BACKCHANNEL_LOGOUT_EVENT = 'http://schemas.openid.net/event/backchannel-logout'
_LOGOUT_TOKEN_TYPES = frozenset({'logout+jwt', 'application/logout+jwt'})
class OIDCClient: class OIDCClient:
"""Low-level OIDC client implementing discovery, PKCE, and token validation.""" """Low-level OIDC client implementing discovery, PKCE, and token validation."""
@@ -42,8 +47,14 @@ class OIDCClient:
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
return await client.request(method, url, **kwargs) return await client.request(method, url, **kwargs)
async def build_authorization_url(self) -> tuple[str, str, str, str]: async def build_authorization_url(self, par: PARPolicy = 'never') -> tuple[str, str, str, str]:
"""Return (authorization_url, state, nonce, code_verifier).""" """Return (authorization_url, state, nonce, code_verifier).
When ``par`` selects Pushed Authorization Requests (RFC 9126), the
authorization request parameters are POSTed to the provider's PAR
endpoint first and the returned URL carries only ``client_id`` and
``request_uri``.
"""
metadata = await self._ensure_metadata() metadata = await self._ensure_metadata()
authorization_endpoint = metadata.get('authorization_endpoint') authorization_endpoint = metadata.get('authorization_endpoint')
if not isinstance(authorization_endpoint, str): if not isinstance(authorization_endpoint, str):
@@ -54,7 +65,6 @@ class OIDCClient:
code_verifier, code_challenge = generate_pkce() code_verifier, code_challenge = generate_pkce()
params = { params = {
'client_id': self._config.client_id,
'response_type': 'code', 'response_type': 'code',
'scope': ' '.join(self._config.scopes), 'scope': ' '.join(self._config.scopes),
'redirect_uri': self._config.redirect_uri, 'redirect_uri': self._config.redirect_uri,
@@ -63,9 +73,67 @@ class OIDCClient:
'code_challenge': code_challenge, 'code_challenge': code_challenge,
'code_challenge_method': 'S256', 'code_challenge_method': 'S256',
} }
url = authorization_endpoint + '?' + urlencode(params)
if self._use_par(metadata, par):
request_uri = await self._push_authorization_request(metadata, params)
redirect_params = {'client_id': self._config.client_id, 'request_uri': request_uri}
else:
redirect_params = {'client_id': self._config.client_id, **params}
url = authorization_endpoint + '?' + urlencode(redirect_params)
return url, state, nonce, code_verifier return url, state, nonce, code_verifier
def _use_par(self, metadata: Mapping[str, Any], policy: PARPolicy) -> bool:
endpoint = metadata.get('pushed_authorization_request_endpoint')
required = metadata.get('require_pushed_authorization_requests') is True
if required:
if not isinstance(endpoint, str):
raise RuntimeError(
'OIDC provider requires pushed authorization requests but does not '
'advertise a pushed_authorization_request_endpoint'
)
if policy == 'never':
raise RuntimeError(
"OIDC provider requires pushed authorization requests; set par to "
"'auto' or 'always'"
)
return True
if policy == 'always':
if not isinstance(endpoint, str):
raise RuntimeError(
'OIDC provider does not advertise a pushed_authorization_request_endpoint'
)
return True
return policy == 'auto' and isinstance(endpoint, str)
async def _push_authorization_request(
self, metadata: Mapping[str, Any], params: Mapping[str, str]
) -> str:
endpoint = metadata.get('pushed_authorization_request_endpoint')
if not isinstance(endpoint, str):
raise RuntimeError(
'OIDC provider does not advertise a pushed_authorization_request_endpoint'
)
# RFC 9126 section 2.1: client_id is required in the body even when the
# client authenticates with the Authorization header.
data = {'client_id': self._config.client_id, **params}
resp = await self._request('POST', endpoint, data=data, auth=self._client_auth())
resp.raise_for_status()
body = resp.json()
request_uri = body.get('request_uri') if isinstance(body, Mapping) else None
if not isinstance(request_uri, str):
raise RuntimeError('PAR endpoint response is missing request_uri')
return request_uri
def _client_auth(self) -> Optional[tuple[str, str]]:
if self._config.client_secret is None:
return None
return (self._config.client_id, self._config.client_secret)
async def fetch_token(self, code: str, code_verifier: str) -> Mapping[str, Any]: async def fetch_token(self, code: str, code_verifier: str) -> Mapping[str, Any]:
metadata = await self._ensure_metadata() metadata = await self._ensure_metadata()
token_endpoint = metadata.get('token_endpoint') token_endpoint = metadata.get('token_endpoint')
@@ -79,11 +147,8 @@ class OIDCClient:
'client_id': self._config.client_id, 'client_id': self._config.client_id,
'code_verifier': code_verifier, 'code_verifier': code_verifier,
} }
auth: Optional[tuple[str, str]] = None
if self._config.client_secret is not None:
auth = (self._config.client_id, self._config.client_secret)
resp = await self._request('POST', token_endpoint, data=data, auth=auth) resp = await self._request('POST', token_endpoint, data=data, auth=self._client_auth())
resp.raise_for_status() resp.raise_for_status()
return cast(Mapping[str, Any], resp.json()) return cast(Mapping[str, Any], resp.json())
@@ -98,15 +163,89 @@ class OIDCClient:
'refresh_token': refresh_token, 'refresh_token': refresh_token,
'client_id': self._config.client_id, 'client_id': self._config.client_id,
} }
auth: Optional[tuple[str, str]] = None
if self._config.client_secret is not None:
auth = (self._config.client_id, self._config.client_secret)
resp = await self._request('POST', token_endpoint, data=data, auth=auth) resp = await self._request('POST', token_endpoint, data=data, auth=self._client_auth())
resp.raise_for_status() resp.raise_for_status()
return cast(Mapping[str, Any], resp.json()) return cast(Mapping[str, Any], resp.json())
async def validate_id_token(self, id_token: str, nonce: str) -> Mapping[str, Any]: async def validate_id_token(self, id_token: str, nonce: str) -> Mapping[str, Any]:
payload = await self._decode_id_token(id_token)
if payload.get('nonce') != nonce:
raise ValueError('ID token nonce mismatch')
return payload
async def validate_refreshed_id_token(
self,
id_token: str,
expected_nonce: Optional[str] = None,
) -> Mapping[str, Any]:
"""Validate an ID token returned by the refresh token grant.
Unlike the authorization code flow, the provider may omit the ``nonce``
claim; when it is present and the original nonce is known, the two must
match (OIDC Core 12.2).
"""
payload = await self._decode_id_token(id_token)
token_nonce = payload.get('nonce')
if token_nonce is not None and expected_nonce is not None and token_nonce != expected_nonce:
raise ValueError('ID token nonce mismatch')
return payload
async def validate_logout_token(self, logout_token: str) -> Mapping[str, Any]:
"""Validate an OIDC Back-Channel Logout token and return its claims."""
header = jwt.get_unverified_header(logout_token)
typ = header.get('typ')
if not isinstance(typ, str) or typ.lower() not in _LOGOUT_TOKEN_TYPES:
raise ValueError('Logout token typ must be logout+jwt')
alg = header.get('alg')
if not isinstance(alg, str):
raise ValueError('Logout token header is missing alg')
if alg == 'none' or alg.startswith('HS'):
raise ValueError('Logout tokens must use an asymmetric signature')
if alg not in self._config.allowed_id_token_algorithms:
raise ValueError(f'Logout token algorithm {alg} is not allowed')
metadata_issuer = await self._get_metadata_issuer()
jwks = await self._ensure_jwks()
signing_key = self._get_signing_key(jwks, header.get('kid'), alg)
payload = jwt.decode(
logout_token,
signing_key.key,
algorithms=[alg],
audience=self._config.client_id,
leeway=self._config.backchannel_logout_leeway,
options={
'require': ['iss', 'aud', 'iat', 'jti', 'events'],
'verify_exp': True,
'verify_iat': True,
'verify_nbf': True,
},
)
if not isinstance(payload, dict):
raise ValueError('Logout token payload is not a JSON object')
self._validate_issuer(payload, metadata_issuer)
if not isinstance(payload.get('jti'), str):
raise ValueError('Logout token jti must be a string')
events = payload.get('events')
if (not isinstance(events, Mapping)
or not isinstance(events.get(_BACKCHANNEL_LOGOUT_EVENT), Mapping)):
raise ValueError('Logout token is missing the back-channel logout event')
if 'nonce' in payload:
raise ValueError('Logout token must not contain a nonce')
sub = payload.get('sub')
sid = payload.get('sid')
if sub is not None and not isinstance(sub, str):
raise ValueError('Logout token sub must be a string')
if sid is not None and not isinstance(sid, str):
raise ValueError('Logout token sid must be a string')
if sub is None and sid is None:
raise ValueError('Logout token must contain sub or sid')
return payload
async def _decode_id_token(self, id_token: str) -> Mapping[str, Any]:
header = jwt.get_unverified_header(id_token) header = jwt.get_unverified_header(id_token)
alg = header.get('alg') alg = header.get('alg')
if not isinstance(alg, str): if not isinstance(alg, str):
@@ -115,29 +254,59 @@ class OIDCClient:
if alg == 'none': if alg == 'none':
if not self._config.allow_unsigned_id_tokens: if not self._config.allow_unsigned_id_tokens:
raise ValueError('Unsigned ID tokens are not allowed') raise ValueError('Unsigned ID tokens are not allowed')
metadata_issuer = await self._get_metadata_issuer()
payload = jwt.decode( payload = jwt.decode(
id_token, id_token,
options={'verify_signature': False}, audience=self._config.client_id,
options={
'verify_signature': False,
'verify_aud': True,
'verify_exp': True,
'verify_iat': True,
'verify_nbf': True,
},
) )
else: else:
if alg not in self._config.allowed_id_token_algorithms: if alg not in self._config.allowed_id_token_algorithms:
raise ValueError(f'ID token algorithm {alg} is not allowed') raise ValueError(f'ID token algorithm {alg} is not allowed')
metadata_issuer = await self._get_metadata_issuer()
jwks = await self._ensure_jwks() jwks = await self._ensure_jwks()
signing_key = self._get_signing_key(jwks, header.get('kid'), alg) signing_key = self._get_signing_key(jwks, header.get('kid'), alg)
payload = jwt.decode( payload = jwt.decode(
id_token, id_token,
signing_key.key, signing_key.key,
algorithms=[alg], algorithms=[alg],
issuer=self._config.issuer,
audience=self._config.client_id, audience=self._config.client_id,
) )
if not isinstance(payload, dict): if not isinstance(payload, dict):
raise ValueError('ID token payload is not a JSON object') raise ValueError('ID token payload is not a JSON object')
if payload.get('nonce') != nonce: self._validate_issuer(payload, metadata_issuer)
raise ValueError('ID token nonce mismatch')
return payload return payload
async def _get_metadata_issuer(self) -> str:
metadata = await self._ensure_metadata()
issuer = metadata.get('issuer')
if not isinstance(issuer, str):
raise RuntimeError('OIDC provider does not advertise an issuer')
return issuer
@staticmethod
def _validate_issuer(payload: Mapping[str, Any], metadata_issuer: str) -> None:
iss = payload.get('iss')
if not isinstance(iss, str):
raise jwt.MissingRequiredClaimError('iss')
expected = metadata_issuer
if _TENANT_ID_PLACEHOLDER.search(expected):
tenant_id = payload.get('tid')
if not isinstance(tenant_id, str):
raise jwt.InvalidIssuerError('Invalid issuer')
expected = _TENANT_ID_PLACEHOLDER.sub(tenant_id, expected, count=1)
if iss != expected:
raise jwt.InvalidIssuerError('Invalid issuer')
def _get_signing_key(self, jwks: Mapping[str, Any], kid: Optional[str], alg: str) -> PyJWK: def _get_signing_key(self, jwks: Mapping[str, Any], kid: Optional[str], alg: str) -> PyJWK:
keys = jwks.get('keys') keys = jwks.get('keys')
if not isinstance(keys, Sequence): if not isinstance(keys, Sequence):
+45 -1
View File
@@ -1,13 +1,37 @@
import re
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Optional, Sequence from typing import Any, Literal, Optional, Sequence
import httpx import httpx
_PROVIDER_PATTERN = re.compile(r'^[A-Za-z0-9_-]+$')
type PARPolicy = Literal['never', 'auto', 'always']
_PAR_POLICIES: tuple[PARPolicy, ...] = ('never', 'auto', 'always')
@dataclass @dataclass
class OIDCConfig: class OIDCConfig:
"""Configuration for an OpenID Connect provider. """Configuration for an OpenID Connect provider.
``provider`` is a stable identifier for this provider. It namespaces the
session state (``oidc:{provider}:...``) so several providers can be used in
the same application without interfering with each other.
``par`` selects when to use Pushed Authorization Requests (RFC 9126):
``'never'`` uses a plain query-string authorization request, ``'auto'``
pushes the request whenever the provider advertises a
``pushed_authorization_request_endpoint``, and ``'always'`` requires PAR.
A provider that sets ``require_pushed_authorization_requests`` is always
treated as requiring PAR.
``backchannel_logout_path`` enables OIDC Back-Channel Logout when set to
an application path. Register the corresponding absolute URL with the
provider. It requires a session store with index support.
``backchannel_logout_leeway`` is the clock-skew allowance, in seconds,
used when validating logout tokens.
``client_secret`` is optional; omit it for public clients. ``client_secret`` is optional; omit it for public clients.
``fetch_userinfo`` controls whether the userinfo endpoint is queried after ``fetch_userinfo`` controls whether the userinfo endpoint is queried after
token exchange. token exchange.
@@ -35,8 +59,28 @@ class OIDCConfig:
'EdDSA', 'EdDSA',
)) ))
http_client: Optional[httpx.AsyncClient] = None http_client: Optional[httpx.AsyncClient] = None
provider: str = 'default'
par: PARPolicy = 'never'
backchannel_logout_path: Optional[str] = None
backchannel_logout_leeway: int = 60
def __post_init__(self) -> None: def __post_init__(self) -> None:
if not self.provider or _PROVIDER_PATTERN.match(self.provider) is None:
raise ValueError(
'provider must be a non-empty string of letters, digits, '
"underscores or hyphens (got "
f'{self.provider!r})'
)
if self.par not in _PAR_POLICIES:
raise ValueError(
f'par must be one of {", ".join(map(repr, _PAR_POLICIES))} '
f'(got {self.par!r})'
)
if (self.backchannel_logout_path is not None
and not self.backchannel_logout_path.startswith('/')):
raise ValueError('backchannel_logout_path must start with "/"')
if self.backchannel_logout_leeway < 0:
raise ValueError('backchannel_logout_leeway must be non-negative')
scopes_list = list(self.scopes) scopes_list = list(self.scopes)
if 'openid' not in scopes_list: if 'openid' not in scopes_list:
self.scopes = ('openid', *scopes_list) self.scopes = ('openid', *scopes_list)
+321 -27
View File
@@ -1,5 +1,6 @@
from typing import Any, Awaitable, Callable, Mapping, Optional, Sequence from typing import Any, Awaitable, Callable, Literal, Mapping, Optional, Sequence, cast
import jwt
from kaya.core import HttpContext, KayaApp, KayaMixin from kaya.core import HttpContext, KayaApp, KayaMixin
from kaya.session import Session, SessionMixin from kaya.session import Session, SessionMixin
from urllib.parse import parse_qs from urllib.parse import parse_qs
@@ -10,6 +11,14 @@ from ._config import OIDCConfig
type HttpHandler = Callable[..., Awaitable[None]] type HttpHandler = Callable[..., Awaitable[None]]
type IdentityStrategy = Literal['single-identity', 'multi-identity']
_SESSION_NAMESPACE = 'oidc'
_USER_SUFFIX = 'user'
_IDENTITY_FIELDS = frozenset({'user', 'id_token', 'access_token', 'refresh_token', 'sid'})
_IDENTITY_STRATEGIES: tuple[IdentityStrategy, ...] = ('single-identity', 'multi-identity')
_BACKCHANNEL_BODY_LIMIT = 64 * 1024
class OIDCUser(Mapping[str, Any]): class OIDCUser(Mapping[str, Any]):
"""Read-only view of the OIDC userinfo / ID token claims.""" """Read-only view of the OIDC userinfo / ID token claims."""
@@ -28,7 +37,7 @@ class OIDCUser(Mapping[str, Any]):
@property @property
def sub(self) -> str: def sub(self) -> str:
return self._data['sub'] return cast(str, self._data['sub'])
@property @property
def email(self) -> Optional[str]: def email(self) -> Optional[str]:
@@ -56,6 +65,21 @@ class OIDCMixin(KayaMixin):
- ``callback_path`` (default ``/auth/callback``): handles the provider callback. - ``callback_path`` (default ``/auth/callback``): handles the provider callback.
- ``logout_path`` (default ``/auth/logout``): logs the user out. - ``logout_path`` (default ``/auth/logout``): logs the user out.
When ``backchannel_logout_path`` is configured, a POST route accepting
OIDC Back-Channel Logout tokens is registered as well. This requires an
index-capable session store.
Session state is namespaced by ``OIDCConfig.provider`` as
``oidc:{provider}:...`` so several providers can share one session. Use one
``OIDCMixin`` per provider (with distinct route paths), or
:class:`~kaya.oidc.OIDCProviders` to wire several at once.
``strategy`` controls how many identities may coexist in the session. The
default ``'single-identity'`` evicts every other provider's identity after a
successful login, so the session has exactly one user (last login wins).
``'multi-identity'`` keeps each provider logged in independently, which is
useful for linked accounts, provider API aggregation, and step-up flows.
Example:: Example::
session = SessionMixin(InMemorySessionStore()) session = SessionMixin(InMemorySessionStore())
@@ -69,23 +93,115 @@ class OIDCMixin(KayaMixin):
... ...
""" """
def __init__(self, config: OIDCConfig, session: SessionMixin) -> None: def __init__(
self,
config: OIDCConfig,
session: SessionMixin,
strategy: IdentityStrategy = 'single-identity',
) -> None:
if strategy not in _IDENTITY_STRATEGIES:
raise ValueError(
'strategy must be one of '
f'{", ".join(map(repr, _IDENTITY_STRATEGIES))} (got {strategy!r})'
)
self._config = config self._config = config
self._session = session self._session = session
self._client = OIDCClient(config) self._client = OIDCClient(config)
self._strategy = strategy
self._prefix = f'{_SESSION_NAMESPACE}:{config.provider}:'
if (config.backchannel_logout_path is not None
and not session.store.supports_indexes):
raise ValueError(
'backchannel_logout_path requires a session store with index support'
)
@property
def provider(self) -> str:
return self._config.provider
@property
def strategy(self) -> IdentityStrategy:
return self._strategy
def _key(self, name: str) -> str:
return f'{self._prefix}{name}'
def _index_name(self, claim: str, provider: Optional[str] = None) -> str:
return f'{_SESSION_NAMESPACE}:{provider or self.provider}:{claim}'
def _apply_identity_indexes(
self,
session: Session,
sub: str,
sid: Optional[str],
) -> None:
session.add_index(self._index_name('sub'), sub)
if sid is not None:
session.add_index(self._index_name('sid'), sid)
session[self._key('indexed')] = True
def _remove_identity_indexes(
self,
session: Session,
provider: Optional[str] = None,
) -> None:
prefix = f'{_SESSION_NAMESPACE}:{provider or self.provider}:'
user = session.get(f'{prefix}user')
if isinstance(user, Mapping):
sub = user.get('sub')
if isinstance(sub, str):
session.remove_index(self._index_name('sub', provider), sub)
sid = session.get(f'{prefix}sid')
if isinstance(sid, str):
session.remove_index(self._index_name('sid', provider), sid)
def _backfill_identity_indexes(self, session: Session) -> None:
"""Index an authenticated session created before index support."""
user = session.get(self._key('user'))
if not isinstance(user, Mapping) or self._key('indexed') in session:
return
sub = user.get('sub')
if not isinstance(sub, str):
return
sid = session.get(self._key('sid'))
if not isinstance(sid, str):
id_token = session.get(self._key('id_token'))
if isinstance(id_token, str):
try:
claims = jwt.decode(
id_token,
options={'verify_signature': False},
)
candidate = claims.get('sid')
if isinstance(candidate, str):
sid = candidate
session[self._key('sid')] = candidate
except jwt.PyJWTError:
pass
self._apply_identity_indexes(
session,
sub,
sid if isinstance(sid, str) else None,
)
@property @property
def dependencies(self) -> Sequence[KayaMixin]: def dependencies(self) -> Sequence[KayaMixin]:
return [self._session] return [self._session]
def apply(self, app: KayaApp) -> None: def apply(self, app: KayaApp) -> None:
if self._config.backchannel_logout_path is not None:
app.add_before_request_hook(self._before_request)
@app.GET(self._config.login_path) @app.GET(self._config.login_path)
async def login(ctx: HttpContext) -> None: async def login(ctx: HttpContext) -> None:
auth_url, state, nonce, code_verifier = await self._client.build_authorization_url() auth_url, state, nonce, code_verifier = await self._client.build_authorization_url(
par=self._config.par
)
session = self._session_of(ctx) session = self._session_of(ctx)
session['oidc_state'] = state session[self._key('state')] = state
session['oidc_nonce'] = nonce session[self._key('nonce')] = nonce
session['oidc_code_verifier'] = code_verifier session[self._key('code_verifier')] = code_verifier
await ctx.send_empty(302, {'Location': auth_url}) await ctx.send_empty(302, {'Location': auth_url})
@app.GET(self._config.callback_path) @app.GET(self._config.callback_path)
@@ -108,13 +224,13 @@ class OIDCMixin(KayaMixin):
return return
session = self._session_of(ctx) session = self._session_of(ctx)
expected_state = session.get('oidc_state') expected_state = session.get(self._key('state'))
if state != expected_state: if state != expected_state:
await ctx.send_str(400, 'Invalid state') await ctx.send_str(400, 'Invalid state')
return return
nonce = session.get('oidc_nonce', '') nonce = session.get(self._key('nonce'), '')
code_verifier = session.get('oidc_code_verifier', '') code_verifier = session.get(self._key('code_verifier'), '')
try: try:
token_response = await self._client.fetch_token(code, code_verifier) token_response = await self._client.fetch_token(code, code_verifier)
@@ -124,6 +240,12 @@ class OIDCMixin(KayaMixin):
return return
claims = await self._client.validate_id_token(id_token, nonce) claims = await self._client.validate_id_token(id_token, nonce)
sub = claims.get('sub')
if not isinstance(sub, str):
raise ValueError('ID token is missing a valid sub claim')
sid = claims.get('sid')
if sid is not None and not isinstance(sid, str):
raise ValueError('ID token sid must be a string')
user: dict[str, Any] = dict(claims) user: dict[str, Any] = dict(claims)
access_token = token_response.get('access_token') access_token = token_response.get('access_token')
@@ -131,31 +253,151 @@ class OIDCMixin(KayaMixin):
userinfo = await self._client.fetch_userinfo(access_token) userinfo = await self._client.fetch_userinfo(access_token)
user.update(userinfo) user.update(userinfo)
session.pop('oidc_state', None) session.pop(self._key('state'), None)
session.pop('oidc_nonce', None) session.pop(self._key('code_verifier'), None)
session.pop('oidc_code_verifier', None) # Keep this provider's nonce so refreshed ID tokens can be
# checked for nonce continuity (OIDC Core 12.2).
session['oidc_user'] = user self._remove_identity_indexes(session)
session['oidc_id_token'] = id_token session[self._key('user')] = user
session[self._key('id_token')] = id_token
session.pop(self._key('sid'), None)
if sid is not None:
session[self._key('sid')] = sid
self._apply_identity_indexes(session, sub, sid)
if isinstance(access_token, str): if isinstance(access_token, str):
session['oidc_access_token'] = access_token session[self._key('access_token')] = access_token
if 'refresh_token' in token_response: if 'refresh_token' in token_response:
session['oidc_refresh_token'] = token_response['refresh_token'] session[self._key('refresh_token')] = token_response['refresh_token']
if self._strategy == 'single-identity':
self._strip_other_identities(session)
session.regenerate_id() session.regenerate_id()
await ctx.send_empty(302, {'Location': self._config.post_login_redirect}) await ctx.send_empty(302, {'Location': self._config.post_login_redirect})
except ValueError as exc: except (ValueError, jwt.PyJWTError) as exc:
await ctx.send_str(400, f'Authentication failed: {exc}') await ctx.send_str(400, f'Authentication failed: {exc}')
@app.GET(self._config.logout_path) @app.GET(self._config.logout_path)
async def logout(ctx: HttpContext) -> None: async def logout(ctx: HttpContext) -> None:
session = self._session_of(ctx) session = self._session_of(ctx)
id_token = session.get('oidc_id_token') id_token = session.get(self._key('id_token'))
session.invalidate() self._logout_session(session)
logout_url = await self._client.build_logout_url(id_token if isinstance(id_token, str) else None) logout_url = await self._client.build_logout_url(id_token if isinstance(id_token, str) else None)
location = logout_url if logout_url is not None else self._config.post_logout_redirect location = logout_url if logout_url is not None else self._config.post_logout_redirect
await ctx.send_empty(302, {'Location': location}) await ctx.send_empty(302, {'Location': location})
backchannel_path = self._config.backchannel_logout_path
if backchannel_path is not None:
@app.POST(backchannel_path)
async def backchannel_logout(ctx: HttpContext) -> None:
# This endpoint is server-to-server. Never persist a browser
# session that happens to accompany the request.
ctx.session = Session()
try:
logout_token = await self._read_logout_token(ctx)
claims = await self._client.validate_logout_token(logout_token)
except (ValueError, jwt.PyJWTError):
await ctx.send_str(
400,
'Invalid logout token',
{'Cache-Control': 'no-store'},
)
return
await self._invalidate_by_logout_token(claims)
await ctx.send_empty(200, {'Cache-Control': 'no-store'})
def _logout_session(self, session: Session) -> None:
"""Drop this provider's session state.
The shared session is only invalidated when no other provider remains
logged in, so logging out of one provider does not sign the user out of
the others.
"""
self._remove_identity_indexes(session)
for key in [key for key in session if key.startswith(self._prefix)]:
del session[key]
if not any(_is_user_key(key) for key in session):
session.invalidate()
def _strip_other_identities(self, session: Session) -> None:
"""Remove every other provider's identity and tokens from the session.
Only identity keys are dropped; ephemeral flow keys (``state``,
``nonce``, ``code_verifier``) are left alone so logins already in flight
can still complete. Under this strategy the last callback to finish
wins.
"""
other_providers = {
_provider_of_user_key(key)
for key in session
if _is_user_key(key) and not key.startswith(self._prefix)
}
for provider in other_providers:
self._remove_identity_indexes(session, provider)
for key in [key for key in session
if _is_identity_key(key) and not key.startswith(self._prefix)]:
del session[key]
async def _before_request(self, ctx: HttpContext) -> Optional[HttpContext]:
session = ctx.session
if isinstance(session, Session):
self._backfill_identity_indexes(session)
return None
async def _read_logout_token(self, ctx: HttpContext) -> str:
content_type_values = ctx.headers.get('content-type')
if isinstance(content_type_values, str):
content_type = content_type_values
elif content_type_values:
content_type = content_type_values[0]
else:
content_type = ''
if content_type.split(';', 1)[0].strip().lower() != 'application/x-www-form-urlencoded':
raise ValueError('Back-channel logout requires a form-encoded request')
body = bytearray()
async for chunk in ctx.request_body:
body.extend(chunk)
if len(body) > _BACKCHANNEL_BODY_LIMIT:
raise ValueError('Back-channel logout request is too large')
form = parse_qs(body.decode('utf-8'), keep_blank_values=True)
logout_token = self._first_value(form.get('logout_token'))
if logout_token is None:
raise ValueError('Missing logout_token')
return logout_token
async def _invalidate_by_logout_token(self, claims: Mapping[str, Any]) -> None:
store = self._session.store
sub = claims.get('sub')
sid = claims.get('sid')
if isinstance(sid, str):
session_ids = await store.find_sessions(self._index_name('sid'), sid)
elif isinstance(sub, str):
session_ids = await store.find_sessions(self._index_name('sub'), sub)
else:
return
for session_id in session_ids:
session = await store.load(session_id)
if session is None:
continue
if isinstance(sid, str) and isinstance(sub, str):
user = session.get(self._key('user'))
if not isinstance(user, Mapping) or user.get('sub') != sub:
continue
# A session can hold several provider identities. Remove only this
# provider's identity and delete the whole session when it was the
# last one, mirroring the local logout behavior.
self._logout_session(session)
if session.invalidated:
await store.delete(session_id)
else:
await store.save(session_id, session, self._session.max_age)
session.commit_indexes()
@staticmethod @staticmethod
def _session_of(ctx: HttpContext) -> Session: def _session_of(ctx: HttpContext) -> Session:
session = ctx.session session = ctx.session
@@ -169,14 +411,31 @@ class OIDCMixin(KayaMixin):
return None return None
def is_authenticated(self, ctx: HttpContext) -> bool: def is_authenticated(self, ctx: HttpContext) -> bool:
return 'oidc_user' in self._session_of(ctx) return self._key('user') in self._session_of(ctx)
def get_user(self, ctx: HttpContext) -> Optional[OIDCUser]: def get_user(self, ctx: HttpContext) -> Optional[OIDCUser]:
user = self._session_of(ctx).get('oidc_user') user = self._session_of(ctx).get(self._key('user'))
if user is None or not isinstance(user, Mapping): if user is None or not isinstance(user, Mapping):
return None return None
return OIDCUser(user) return OIDCUser(user)
def logged_in_providers(self, ctx: HttpContext) -> list[str]:
"""Return the names of every provider with a user in this session."""
return [_provider_of_user_key(key)
for key in self._session_of(ctx) if _is_user_key(key)]
def get_users(self, ctx: HttpContext) -> dict[str, OIDCUser]:
"""Return the users of every provider logged into this session."""
session = self._session_of(ctx)
users: dict[str, OIDCUser] = {}
for key in session:
if not _is_user_key(key):
continue
data = session.get(key)
if isinstance(data, Mapping):
users[_provider_of_user_key(key)] = OIDCUser(data)
return users
def require_auth(self, handler: HttpHandler) -> HttpHandler: def require_auth(self, handler: HttpHandler) -> HttpHandler:
async def wrapper(ctx: HttpContext, *args: Any, **kwargs: Any) -> None: async def wrapper(ctx: HttpContext, *args: Any, **kwargs: Any) -> None:
if not self.is_authenticated(ctx): if not self.is_authenticated(ctx):
@@ -186,18 +445,53 @@ class OIDCMixin(KayaMixin):
return wrapper return wrapper
async def refresh_access_token(self, session: Session) -> Optional[str]: async def refresh_access_token(self, session: Session) -> Optional[str]:
refresh_token = session.get('oidc_refresh_token') refresh_token = session.get(self._key('refresh_token'))
if not isinstance(refresh_token, str): if not isinstance(refresh_token, str):
return None return None
try: try:
token_response = await self._client.refresh_token(refresh_token) token_response = await self._client.refresh_token(refresh_token)
access_token = token_response.get('access_token') access_token = token_response.get('access_token')
id_token = token_response.get('id_token')
if isinstance(id_token, str):
stored_nonce = session.get(self._key('nonce'))
claims = await self._client.validate_refreshed_id_token(
id_token,
stored_nonce if isinstance(stored_nonce, str) else None,
)
user = session.get(self._key('user'))
original_sub = user.get('sub') if isinstance(user, Mapping) else None
if isinstance(original_sub, str) and claims.get('sub') != original_sub:
raise ValueError('ID token subject mismatch')
refreshed_sid = claims.get('sid')
stored_sid = session.get(self._key('sid'))
if (isinstance(refreshed_sid, str)
and refreshed_sid != stored_sid):
if isinstance(stored_sid, str):
session.remove_index(self._index_name('sid'), stored_sid)
session[self._key('sid')] = refreshed_sid
session.add_index(self._index_name('sid'), refreshed_sid)
if isinstance(access_token, str): if isinstance(access_token, str):
session['oidc_access_token'] = access_token session[self._key('access_token')] = access_token
if 'id_token' in token_response: if isinstance(id_token, str):
session['oidc_id_token'] = token_response['id_token'] session[self._key('id_token')] = id_token
if 'refresh_token' in token_response: if 'refresh_token' in token_response:
session['oidc_refresh_token'] = token_response['refresh_token'] session[self._key('refresh_token')] = token_response['refresh_token']
return access_token if isinstance(access_token, str) else None return access_token if isinstance(access_token, str) else None
except Exception: except Exception:
return None return None
def _is_user_key(key: str) -> bool:
return key.startswith(f'{_SESSION_NAMESPACE}:') and key.endswith(f':{_USER_SUFFIX}')
def _is_identity_key(key: str) -> bool:
return (key.startswith(f'{_SESSION_NAMESPACE}:')
and key.rsplit(':', 1)[-1] in _IDENTITY_FIELDS)
def _provider_of_user_key(key: str) -> str:
return key[len(_SESSION_NAMESPACE) + 1:-(len(_USER_SUFFIX) + 1)]
@@ -0,0 +1,141 @@
from dataclasses import replace
from types import MappingProxyType
from typing import Callable, Mapping, Optional, Sequence
from kaya.core import HttpContext, KayaApp, KayaMixin
from kaya.session import Session, SessionMixin
from ._config import OIDCConfig
from ._mixin import (
_IDENTITY_STRATEGIES,
HttpHandler,
IdentityStrategy,
OIDCMixin,
OIDCUser,
)
class OIDCProviders(KayaMixin):
"""Wire several OpenID Connect providers into one application.
Each config's ``provider`` becomes a route segment under ``base_path``:
``{base_path}/{provider}/login``, ``{base_path}/{provider}/callback`` and
``{base_path}/{provider}/logout``. Providers that enable back-channel
logout also get ``{base_path}/{provider}/backchannel-logout``. All
providers share the supplied
:class:`~kaya.session.SessionMixin` and namespace their session state by
provider.
``strategy`` controls how many identities may coexist in the session. The
default ``'single-identity'`` evicts every other provider's identity after a
successful login (last login wins). ``'multi-identity'`` keeps each provider
logged in independently, so logging out of one provider does not sign the
user out of the others.
Example::
session = SessionMixin(InMemorySessionStore())
oidc = OIDCProviders([google_config, github_config], session=session)
app = KayaApp(mixins=[session, oidc])
@app.GET('/profile')
@oidc.require_auth('google')
async def profile(ctx: HttpContext):
user = oidc.get_user(ctx, 'google')
...
"""
def __init__(
self,
configs: Sequence[OIDCConfig],
session: SessionMixin,
base_path: str = '/auth',
strategy: IdentityStrategy = 'single-identity',
) -> None:
if not configs:
raise ValueError('At least one OIDCConfig is required')
if not base_path.startswith('/'):
raise ValueError('base_path must start with "/"')
if strategy not in _IDENTITY_STRATEGIES:
raise ValueError(
'strategy must be one of '
f'{", ".join(map(repr, _IDENTITY_STRATEGIES))} (got {strategy!r})'
)
self._session = session
self._strategy = strategy
self._base_path = base_path.rstrip('/')
self._providers: dict[str, OIDCMixin] = {}
for config in configs:
name = config.provider
if name in self._providers:
raise ValueError(f'Duplicate OIDC provider {name!r}')
backchannel_path = (
f'{self._base_path}/{name}/backchannel-logout'
if config.backchannel_logout_path is not None else None
)
self._providers[name] = OIDCMixin(
replace(
config,
login_path=f'{self._base_path}/{name}/login',
callback_path=f'{self._base_path}/{name}/callback',
logout_path=f'{self._base_path}/{name}/logout',
backchannel_logout_path=backchannel_path,
),
session=session,
strategy=strategy,
)
@property
def base_path(self) -> str:
return self._base_path
@property
def strategy(self) -> IdentityStrategy:
return self._strategy
@property
def providers(self) -> Mapping[str, OIDCMixin]:
return MappingProxyType(self._providers)
def provider(self, name: str) -> OIDCMixin:
try:
return self._providers[name]
except KeyError:
raise KeyError(f'Unknown OIDC provider {name!r}') from None
def __getitem__(self, name: str) -> OIDCMixin:
return self.provider(name)
@property
def dependencies(self) -> Sequence[KayaMixin]:
return [self._session]
def apply(self, app: KayaApp) -> None:
for mixin in self._providers.values():
mixin.apply(app)
def is_authenticated(self, ctx: HttpContext, provider: str) -> bool:
return self.provider(provider).is_authenticated(ctx)
def get_user(self, ctx: HttpContext, provider: str) -> Optional[OIDCUser]:
return self.provider(provider).get_user(ctx)
def get_users(self, ctx: HttpContext) -> dict[str, OIDCUser]:
"""Return the user of every provider with a session on ``ctx``."""
users: dict[str, OIDCUser] = {}
for name, mixin in self._providers.items():
user = mixin.get_user(ctx)
if user is not None:
users[name] = user
return users
def logged_in_providers(self, ctx: HttpContext) -> list[str]:
"""Return the names of the providers logged into ``ctx``."""
return [name for name, mixin in self._providers.items() if mixin.is_authenticated(ctx)]
def require_auth(self, provider: str) -> Callable[[HttpHandler], HttpHandler]:
return self.provider(provider).require_auth
async def refresh_access_token(self, session: Session, provider: str) -> Optional[str]:
return await self.provider(provider).refresh_access_token(session)
@@ -0,0 +1,427 @@
import unittest
from time import time
from typing import Any, Optional
from urllib.parse import parse_qs, urlparse
import httpx
import jwt
from cryptography.hazmat.primitives.asymmetric import rsa
from pwo import async_test
from test_oidc import MockOIDCProvider, MockTransport
from kaya.core import HttpContext, KayaApp
from kaya.oidc import OIDCConfig, OIDCMixin, OIDCProviders
from kaya.session import InMemorySessionStore, SessionMixin
_EVENT = 'http://schemas.openid.net/event/backchannel-logout'
class BackchannelProvider(MockOIDCProvider):
def __init__(self, *args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)
self.sid: Optional[str] = None
def issue_id_token(self, *args: Any, **kwargs: Any) -> str:
token = super().issue_id_token(*args, **kwargs)
if self.sid is None:
return token
payload = jwt.decode(token, options={'verify_signature': False})
payload['sid'] = self.sid
return jwt.encode(
payload,
self.private_key,
algorithm='RS256',
headers={'kid': self.kid},
)
class BackchannelLogoutTest(unittest.TestCase):
def setUp(self) -> None:
self.provider = BackchannelProvider()
self.http_client = httpx.AsyncClient(transport=MockTransport(self.provider))
self.store = InMemorySessionStore()
self.session = SessionMixin(self.store)
self.oidc = OIDCMixin(
OIDCConfig(
issuer=self.provider.issuer,
client_id='client',
client_secret='secret',
redirect_uri='http://localhost:8000/auth/callback',
backchannel_logout_path='/auth/backchannel-logout',
http_client=self.http_client,
),
session=self.session,
)
self.app = KayaApp(mixins=[self.session, self.oidc])
self.transport = httpx.ASGITransport(app=self.app)
@self.app.GET('/profile')
@self.oidc.require_auth
async def profile(ctx: HttpContext) -> None:
user = self.oidc.get_user(ctx)
assert user is not None
await ctx.send_str(200, user.sub)
@self.app.GET('/refresh')
@self.oidc.require_auth
async def refresh(ctx: HttpContext) -> None:
token = await self.oidc.refresh_access_token(ctx.session)
await ctx.send_str(200, token or 'no-token')
def _client(self) -> httpx.AsyncClient:
return httpx.AsyncClient(transport=self.transport, base_url='http://127.0.0.1:80')
async def _login(self, client: httpx.AsyncClient, sid: str) -> None:
self.provider.sid = sid
response = await client.get('/auth/login', follow_redirects=False)
self.assertEqual(302, response.status_code)
query = parse_qs(urlparse(response.headers['Location']).query)
self.provider.nonce = query['nonce'][0]
response = await client.get(
'/auth/callback',
params={'code': 'mock-code', 'state': query['state'][0]},
follow_redirects=False,
)
self.assertEqual(302, response.status_code)
def _claims(self, **overrides: Any) -> dict[str, Any]:
claims: dict[str, Any] = {
'iss': self.provider.issuer,
'aud': 'client',
'iat': time(),
'jti': 'logout-token-id',
'events': {_EVENT: {}},
'sub': self.provider.sub,
}
claims.update(overrides)
return claims
def _token(
self,
claims: dict[str, Any],
*,
typ: str = 'logout+jwt',
key: Any = None,
algorithm: str = 'RS256',
) -> str:
return jwt.encode(
claims,
self.provider.private_key if key is None else key,
algorithm=algorithm,
headers={'kid': self.provider.kid, 'typ': typ},
)
async def _post_logout(self, token: str) -> httpx.Response:
async with self._client() as client:
return await client.post(
'/auth/backchannel-logout',
data={'logout_token': token},
)
async def _profile_status(self, client: httpx.AsyncClient) -> int:
response = await client.get('/profile', follow_redirects=False)
return response.status_code
@async_test
async def test_logout_by_sid_only_invalidates_matching_session(self) -> None:
async with self._client() as first, self._client() as second:
await self._login(first, 'sid-a')
await self._login(second, 'sid-b')
response = await self._post_logout(self._token(self._claims(sid='sid-a')))
self.assertEqual(200, response.status_code)
self.assertEqual('no-store', response.headers['Cache-Control'])
self.assertNotIn('Set-Cookie', response.headers)
self.assertEqual(302, await self._profile_status(first))
self.assertEqual(200, await self._profile_status(second))
self.assertEqual([], await self.store.find_sessions('oidc:default:sid', 'sid-a'))
self.assertEqual(
[second.cookies['session_id']],
await self.store.find_sessions('oidc:default:sid', 'sid-b'),
)
@async_test
async def test_logout_by_sub_invalidates_all_user_sessions(self) -> None:
async with self._client() as first, self._client() as second, self._client() as other:
await self._login(first, 'sid-a')
await self._login(second, 'sid-b')
self.provider.sub = 'bob'
await self._login(other, 'sid-c')
response = await self._post_logout(self._token(self._claims(sub='user123')))
self.assertEqual(200, response.status_code)
self.assertEqual(302, await self._profile_status(first))
self.assertEqual(302, await self._profile_status(second))
self.assertEqual(200, await self._profile_status(other))
@async_test
async def test_sid_logout_verifies_sub_when_both_are_present(self) -> None:
async with self._client() as client:
await self._login(client, 'sid-a')
response = await self._post_logout(
self._token(self._claims(sub='someone-else', sid='sid-a'))
)
self.assertEqual(200, response.status_code)
self.assertEqual(200, await self._profile_status(client))
@async_test
async def test_unknown_session_is_idempotent(self) -> None:
response = await self._post_logout(self._token(self._claims(sid='unknown')))
self.assertEqual(200, response.status_code)
@async_test
async def test_invalid_logout_tokens_are_rejected(self) -> None:
other_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
missing_iat = self._claims(sid='sid-a')
del missing_iat['iat']
missing_jti = self._claims(sid='sid-a')
del missing_jti['jti']
missing_events = self._claims(sid='sid-a')
del missing_events['events']
missing_subject = self._claims()
del missing_subject['sub']
invalid_tokens = [
self._token(self._claims(sid='sid-a'), typ='JWT'),
jwt.encode(
self._claims(sid='sid-a'),
None,
algorithm='none',
headers={'typ': 'logout+jwt'},
),
self._token(self._claims(sid='sid-a'), key=other_key),
self._token(self._claims(iss='https://evil.local', sid='sid-a')),
self._token(self._claims(aud='other-client', sid='sid-a')),
self._token(missing_iat),
self._token(missing_jti),
self._token(missing_events),
self._token(self._claims(events={'wrong-event': {}}, sid='sid-a')),
self._token(self._claims(events={_EVENT: None}, sid='sid-a')),
self._token(self._claims(nonce='forbidden', sid='sid-a')),
self._token(missing_subject),
self._token(self._claims(exp=time() - 3600, sid='sid-a')),
self._token(self._claims(sid='sid-a'), key='x' * 32, algorithm='HS256'),
]
async with self._client() as client:
await self._login(client, 'sid-a')
for token in invalid_tokens:
response = await self._post_logout(token)
self.assertEqual(400, response.status_code)
self.assertEqual(200, await self._profile_status(client))
@async_test
async def test_invalid_requests_are_rejected(self) -> None:
async with self._client() as client:
response = await client.post('/auth/backchannel-logout', data={})
self.assertEqual(400, response.status_code)
response = await client.post(
'/auth/backchannel-logout',
content='logout_token=abc',
headers={'Content-Type': 'text/plain'},
)
self.assertEqual(400, response.status_code)
response = await client.post(
'/auth/backchannel-logout',
content='x' * (64 * 1024 + 1),
headers={'Content-Type': 'application/x-www-form-urlencoded'},
)
self.assertEqual(400, response.status_code)
@async_test
async def test_local_logout_removes_indexes(self) -> None:
async with self._client() as client:
await self._login(client, 'sid-a')
self.assertEqual(
[client.cookies['session_id']],
await self.store.find_sessions('oidc:default:sub', 'user123'),
)
await client.get('/auth/logout', follow_redirects=False)
self.assertEqual([], await self.store.find_sessions('oidc:default:sub', 'user123'))
self.assertEqual([], await self.store.find_sessions('oidc:default:sid', 'sid-a'))
@async_test
async def test_relogin_replaces_old_sid_index(self) -> None:
async with self._client() as client:
await self._login(client, 'sid-a')
await self._login(client, 'sid-b')
self.assertEqual([], await self.store.find_sessions('oidc:default:sid', 'sid-a'))
self.assertEqual(
[client.cookies['session_id']],
await self.store.find_sessions('oidc:default:sid', 'sid-b'),
)
@async_test
async def test_refresh_updates_changed_sid_index(self) -> None:
async with self._client() as client:
await self._login(client, 'sid-a')
self.provider.sid = 'sid-b'
response = await client.get('/refresh')
self.assertEqual(200, response.status_code)
self.assertNotEqual('no-token', response.text)
self.assertEqual([], await self.store.find_sessions('oidc:default:sid', 'sid-a'))
self.assertEqual(
[client.cookies['session_id']],
await self.store.find_sessions('oidc:default:sid', 'sid-b'),
)
@async_test
async def test_existing_session_is_backfilled(self) -> None:
async with self._client() as client:
await self._login(client, 'sid-a')
session_id = client.cookies['session_id']
self.store._data[session_id].pop('oidc:default:indexed')
self.store._session_indexes.pop(session_id, None)
for session_ids in self.store._index_sessions.values():
session_ids.discard(session_id)
self.assertEqual([], await self.store.find_sessions('oidc:default:sid', 'sid-a'))
self.assertEqual(200, await self._profile_status(client))
self.assertEqual(
[session_id],
await self.store.find_sessions('oidc:default:sid', 'sid-a'),
)
@async_test
async def test_index_capable_store_is_required(self) -> None:
class LegacyStore(InMemorySessionStore):
supports_indexes = False
with self.assertRaises(ValueError):
OIDCMixin(
OIDCConfig(
issuer=self.provider.issuer,
client_id='client',
redirect_uri='http://localhost:8000/auth/callback',
backchannel_logout_path='/auth/backchannel-logout',
),
session=SessionMixin(LegacyStore()),
)
class MultiProviderBackchannelLogoutTest(unittest.TestCase):
def setUp(self) -> None:
self.google = BackchannelProvider('google', sub='shared-sub', email='g@example.com')
self.github = BackchannelProvider('github', sub='shared-sub', email='h@example.com')
self.store = InMemorySessionStore()
self.session = SessionMixin(self.store)
self.oidc = OIDCProviders(
[
self._config(self.google, 'google'),
self._config(self.github, 'github'),
],
session=self.session,
strategy='multi-identity',
)
self.app = KayaApp(mixins=[self.session, self.oidc])
self.transport = httpx.ASGITransport(app=self.app)
@self.app.GET('/google-profile')
@self.oidc.require_auth('google')
async def google_profile(ctx: HttpContext) -> None:
await ctx.send_str(200, 'google')
@self.app.GET('/github-profile')
@self.oidc.require_auth('github')
async def github_profile(ctx: HttpContext) -> None:
await ctx.send_str(200, 'github')
@staticmethod
def _config(provider: BackchannelProvider, name: str) -> OIDCConfig:
return OIDCConfig(
issuer=provider.issuer,
client_id='client',
client_secret='secret',
redirect_uri=f'http://localhost:8000/auth/{name}/callback',
provider=name,
backchannel_logout_path=f'/auth/{name}/backchannel-logout',
http_client=httpx.AsyncClient(transport=MockTransport(provider)),
)
async def _login(
self,
client: httpx.AsyncClient,
provider: BackchannelProvider,
name: str,
sid: str,
) -> None:
provider.sid = sid
response = await client.get(f'/auth/{name}/login', follow_redirects=False)
query = parse_qs(urlparse(response.headers['Location']).query)
provider.nonce = query['nonce'][0]
await client.get(
f'/auth/{name}/callback',
params={'code': 'mock-code', 'state': query['state'][0]},
follow_redirects=False,
)
@async_test
async def test_single_identity_eviction_removes_old_provider_indexes(self) -> None:
session = SessionMixin(self.store)
oidc = OIDCProviders(
[
self._config(self.google, 'google'),
self._config(self.github, 'github'),
],
session=session,
strategy='single-identity',
)
app = KayaApp(mixins=[session, oidc])
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(
transport=transport,
base_url='http://127.0.0.1:80',
) as client:
await self._login(client, self.google, 'google', 'google-sid')
self.assertEqual(
[client.cookies['session_id']],
await self.store.find_sessions('oidc:google:sid', 'google-sid'),
)
await self._login(client, self.github, 'github', 'github-sid')
self.assertEqual(
[],
await self.store.find_sessions('oidc:google:sid', 'google-sid'),
)
self.assertEqual(
[client.cookies['session_id']],
await self.store.find_sessions('oidc:github:sid', 'github-sid'),
)
@async_test
async def test_provider_logout_is_isolated(self) -> None:
async with httpx.AsyncClient(
transport=self.transport,
base_url='http://127.0.0.1:80',
) as client:
await self._login(client, self.google, 'google', 'google-sid')
await self._login(client, self.github, 'github', 'github-sid')
token = jwt.encode(
{
'iss': self.google.issuer,
'aud': 'client',
'iat': time(),
'jti': 'google-logout',
'events': {_EVENT: {}},
'sub': 'shared-sub',
'sid': 'google-sid',
},
self.google.private_key,
algorithm='RS256',
headers={'kid': self.google.kid, 'typ': 'logout+jwt'},
)
response = await client.post(
'/auth/google/backchannel-logout',
data={'logout_token': token},
)
self.assertEqual(200, response.status_code)
response = await client.get('/google-profile', follow_redirects=False)
self.assertEqual(302, response.status_code)
response = await client.get('/github-profile', follow_redirects=False)
self.assertEqual(200, response.status_code)
if __name__ == '__main__':
unittest.main()
File diff suppressed because it is too large. Load diff
+43
View File
@@ -69,6 +69,48 @@ object, so any valid OpenAPI 3.1 construct can be used:
`parameters` entries whose `name` and `in` match an auto-generated path `parameters` entries whose `name` and `in` match an auto-generated path
parameter override it; all others are appended. parameter override it; all others are appended.
## Reusable schemas (`components`)
Schemas can be defined once and referenced from operation fragments: pass a
`components` mapping to the mixin and use `schema_ref` in `@operation`.
```python
from kaya.openapi import OpenAPIMixin, operation, schema_ref
USER = {
'type': 'object',
'required': ['id'],
'properties': {'id': {'type': 'integer'}, 'name': {'type': 'string'}},
}
USER_PAGE = {
'type': 'object',
'properties': {'results': {'type': 'array', 'items': schema_ref('User')}},
}
app = KayaApp(mixins=[OpenAPIMixin(
title='My API',
version='1.0.0',
components={'schemas': {'User': USER, 'UserPage': USER_PAGE}},
)])
@app.GET('/users')
@operation(summary='List users',
responses={200: {
'description': 'A page of users',
'content': {'application/json': {'schema': schema_ref('UserPage')}},
}})
async def list_users(ctx: HttpContext) -> None:
...
```
`components` is merged verbatim (as a copy) into the generated document under
`components`, so Swagger UI lists the named schemas. Every
`#/components/<section>/<name>` reference — in operations or between
components — is validated when the document is generated: a missing section or
entry raises `ValueError`. Reflected references (`./other.json#/...`, absolute
URLs) are left alone.
## Configuration ## Configuration
```python ```python
@@ -81,6 +123,7 @@ OpenAPIMixin(
servers=[{'url': 'https://api.example.com'}], servers=[{'url': 'https://api.example.com'}],
openapi_version='3.1.0', openapi_version='3.1.0',
include_self=False, # include spec/docs endpoints in the document include_self=False, # include spec/docs endpoints in the document
components=None, # reusable OpenAPI components (e.g. schemas)
) )
``` ```
-5
View File
@@ -25,11 +25,6 @@ dependencies = [
"kaya-core", "kaya-core",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
@@ -1,10 +1,11 @@
from ._metadata import operation from ._metadata import operation
from ._mixin import OpenAPIMixin from ._mixin import OpenAPIMixin
from ._spec import generate_spec from ._spec import generate_spec, schema_ref
__all__ = [ __all__ = [
'OpenAPIMixin', 'OpenAPIMixin',
'generate_spec', 'generate_spec',
'operation', 'operation',
'schema_ref',
] ]
@@ -25,7 +25,9 @@ def operation(summary: Optional[str] = None,
:class:`~kaya.openapi.OpenAPIMixin` when generating the specification. :class:`~kaya.openapi.OpenAPIMixin` when generating the specification.
All schema fragments are plain dicts inserted verbatim into the generated All schema fragments are plain dicts inserted verbatim into the generated
OpenAPI document, so any valid OpenAPI 3.1 construct can be used. OpenAPI document, so any valid OpenAPI 3.1 construct can be used. ``$ref``
fragments may point at the reusable schemas registered through the
``components`` argument of :class:`~kaya.openapi.OpenAPIMixin`.
Example:: Example::
@@ -55,6 +55,11 @@ class OpenAPIMixin(KayaMixin):
bodies and response schemas to individual handlers; the handler docstring bodies and response schemas to individual handlers; the handler docstring
is used as the operation description when no explicit one is given. is used as the operation description when no explicit one is given.
Reusable schemas are supplied through ``components`` and referenced from
operation fragments with :func:`~kaya.openapi.schema_ref`; every
``#/components/...`` reference is validated when the document is
generated.
Example:: Example::
openapi = OpenAPIMixin(title='My API', version='1.0.0') openapi = OpenAPIMixin(title='My API', version='1.0.0')
@@ -74,7 +79,8 @@ class OpenAPIMixin(KayaMixin):
docs_path: str = '/docs', docs_path: str = '/docs',
servers: Optional[Sequence[Mapping[str, Any]]] = None, servers: Optional[Sequence[Mapping[str, Any]]] = None,
openapi_version: str = '3.1.0', openapi_version: str = '3.1.0',
include_self: bool = False) -> None: include_self: bool = False,
components: Optional[Mapping[str, Any]] = None) -> None:
self._title = title self._title = title
self._version = version self._version = version
self._description = description self._description = description
@@ -83,6 +89,7 @@ class OpenAPIMixin(KayaMixin):
self._servers = servers self._servers = servers
self._openapi_version = openapi_version self._openapi_version = openapi_version
self._include_self = include_self self._include_self = include_self
self._components = components
def apply(self, app: KayaApp) -> None: def apply(self, app: KayaApp) -> None:
@app.GET(self._spec_path) @app.GET(self._spec_path)
@@ -107,4 +114,5 @@ class OpenAPIMixin(KayaMixin):
description=self._description, description=self._description,
servers=self._servers, servers=self._servers,
exclude_paths=exclude_paths, exclude_paths=exclude_paths,
openapi_version=self._openapi_version) openapi_version=self._openapi_version,
components=self._components)
@@ -1,7 +1,7 @@
from copy import deepcopy from copy import deepcopy
from dataclasses import dataclass from dataclasses import dataclass
from inspect import getdoc from inspect import getdoc
from typing import AbstractSet, Any, Mapping, Optional, Sequence from typing import AbstractSet, Any, Dict, Iterator, Mapping, Optional, Sequence
from kaya.core import HttpMethod, KayaApp from kaya.core import HttpMethod, KayaApp
from kaya.core._path_handler import PathHandler from kaya.core._path_handler import PathHandler
@@ -22,6 +22,17 @@ STANDARD_METHODS: Sequence[HttpMethod] = (
) )
def schema_ref(name: str) -> Dict[str, str]:
"""Return a reference to the component schema ``name``.
The reference resolves to ``#/components/schemas/<name>`` in the
generated document; the schema itself is supplied through the
``components`` argument of :func:`generate_spec` (or
:class:`~kaya.openapi.OpenAPIMixin`).
"""
return {'$ref': f'#/components/schemas/{name}'}
@dataclass @dataclass
class _Route: class _Route:
raw_path: str raw_path: str
@@ -122,13 +133,67 @@ def _build_operation(params: Sequence[Mapping[str, Any]],
return operation return operation
def _unescape_pointer_token(token: str) -> str:
"""Decode one JSON Pointer reference token (``~1`` → ``/``, ``~0`` → ``~``)."""
return token.replace('~1', '/').replace('~0', '~')
def _iter_refs(node: Any) -> Iterator[str]:
"""Yield every ``$ref`` string found anywhere under ``node``."""
if isinstance(node, Mapping):
ref = node.get('$ref')
if isinstance(ref, str):
yield ref
for value in node.values():
yield from _iter_refs(value)
elif isinstance(node, (list, tuple)):
for value in node:
yield from _iter_refs(value)
def _validate_component_refs(spec: Mapping[str, Any]) -> None:
"""Fail fast on references to missing ``components`` entries.
Every ``#/components/<section>/<name>`` reference found in the operations
or in the components themselves must resolve against the document;
references in any other form (relative files, absolute URLs, other
pointers) are ignored. Raises :class:`ValueError` naming the offending
reference and the operation it appears in.
"""
components = spec.get('components')
known: Mapping[str, Any] = components if isinstance(components, Mapping) else {}
def check(node: Any, location: str) -> None:
for ref in _iter_refs(node):
if not ref.startswith('#/components/'):
continue
parts = [_unescape_pointer_token(part) for part in ref[len('#/components/'):].split('/')]
if len(parts) < 2:
raise ValueError(f'invalid component reference {ref!r} in {location}')
section, name = parts[0], parts[1]
bucket = known.get(section)
if not isinstance(bucket, Mapping) or name not in bucket:
raise ValueError(f'unknown component reference {ref!r} in {location}')
paths = spec.get('paths')
if isinstance(paths, Mapping):
for path, item in paths.items():
if not isinstance(item, Mapping):
continue
for method, operation in item.items():
check(operation, f'{method.upper()} {path}')
if isinstance(components, Mapping):
check(components, 'components')
def generate_spec(app: KayaApp, def generate_spec(app: KayaApp,
title: str, title: str,
version: str, version: str,
description: str = '', description: str = '',
servers: Optional[Sequence[Mapping[str, Any]]] = None, servers: Optional[Sequence[Mapping[str, Any]]] = None,
exclude_paths: AbstractSet[str] = frozenset(), exclude_paths: AbstractSet[str] = frozenset(),
openapi_version: str = '3.1.0') -> Mapping[str, Any]: openapi_version: str = '3.1.0',
components: Optional[Mapping[str, Any]] = None) -> Mapping[str, Any]:
"""Generate an OpenAPI specification document from a :class:`KayaApp`. """Generate an OpenAPI specification document from a :class:`KayaApp`.
The application's routing tree is walked and every route is converted to The application's routing tree is walked and every route is converted to
@@ -147,6 +212,9 @@ def generate_spec(app: KayaApp,
:param servers: list of OpenAPI server objects :param servers: list of OpenAPI server objects
:param exclude_paths: Kaya paths (e.g. ``/openapi.json``) to omit :param exclude_paths: Kaya paths (e.g. ``/openapi.json``) to omit
:param openapi_version: OpenAPI version to declare :param openapi_version: OpenAPI version to declare
:param components: OpenAPI ``components`` object merged verbatim into the
document; references to ``#/components/<section>/<name>`` must resolve
against it (e.g. :func:`schema_ref` for reusable schemas)
:return: the OpenAPI document as a JSON-serializable mapping :return: the OpenAPI document as a JSON-serializable mapping
""" """
routes: list[_Route] = [] routes: list[_Route] = []
@@ -174,4 +242,7 @@ def generate_spec(app: KayaApp,
} }
if servers: if servers:
spec['servers'] = [dict(server) for server in servers] spec['servers'] = [dict(server) for server in servers]
if components:
spec['components'] = deepcopy(dict(components))
_validate_component_refs(spec)
return spec return spec
+123 -1
View File
@@ -6,7 +6,7 @@ import httpx
from pwo import async_test from pwo import async_test
from kaya.core import HttpContext, HttpMethod, KayaApp, WebSocket from kaya.core import HttpContext, HttpMethod, KayaApp, WebSocket
from kaya.openapi import OpenAPIMixin, operation from kaya.openapi import OpenAPIMixin, generate_spec, operation, schema_ref
class OpenAPITest(unittest.TestCase): class OpenAPITest(unittest.TestCase):
@@ -163,5 +163,127 @@ class OpenAPITest(unittest.TestCase):
self.assertIn('/spec.json', response.text) self.assertIn('/spec.json', response.text)
class ComponentsTest(unittest.TestCase):
"""Reusable schemas through ``components`` and ``schema_ref``."""
async def _spec_of(self, app: KayaApp) -> Mapping[str, Any]:
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url='http://127.0.0.1:80') as client:
response = await client.get('/openapi.json')
self.assertEqual(200, response.status_code)
return json.loads(response.text)
def test_schema_ref_helper(self) -> None:
self.assertEqual({'$ref': '#/components/schemas/User'}, schema_ref('User'))
@async_test
async def test_components_merged_and_refs_preserved(self) -> None:
user = {'type': 'object', 'properties': {'id': {'type': 'integer'}}}
page = {'type': 'object',
'properties': {'results': {'type': 'array', 'items': schema_ref('User')}}}
app = KayaApp(mixins=[OpenAPIMixin(
title='Components API',
version='1.0.0',
components={'schemas': {'User': user, 'UserPage': page}},
)])
@app.GET('/users')
@operation(responses={200: {
'description': 'A page of users',
'content': {'application/json': {'schema': schema_ref('UserPage')}},
}})
async def list_users(ctx: HttpContext) -> None:
await ctx.send_str(200, '[]')
spec = await self._spec_of(app)
self.assertEqual({'schemas': {'User': user, 'UserPage': page}}, spec['components'])
schema = spec['paths']['/users']['get']['responses']['200']['content']['application/json']['schema']
self.assertEqual(schema_ref('UserPage'), schema)
@async_test
async def test_request_body_ref_preserved(self) -> None:
app = KayaApp(mixins=[OpenAPIMixin(
title='Components API',
version='1.0.0',
components={'schemas': {'User': {'type': 'object'}}},
)])
@app.POST('/users')
@operation(request_body={'required': True,
'content': {'application/json': {'schema': schema_ref('User')}}},
responses={201: {'description': 'Created'}})
async def create_user(ctx: HttpContext) -> None:
await ctx.send_str(201, '{}')
spec = await self._spec_of(app)
schema = spec['paths']['/users']['post']['requestBody']['content']['application/json']['schema']
self.assertEqual(schema_ref('User'), schema)
@async_test
async def test_default_spec_has_no_components(self) -> None:
app = KayaApp(mixins=[OpenAPIMixin(title='Plain API', version='1.0.0')])
@app.GET('/plain')
async def plain(ctx: HttpContext) -> None:
await ctx.send_str(200, 'ok')
spec = await self._spec_of(app)
self.assertNotIn('components', spec)
def test_unknown_schema_ref_raises(self) -> None:
app = KayaApp()
@app.GET('/broken')
@operation(responses={200: {
'description': 'x',
'content': {'application/json': {'schema': schema_ref('Missing')}},
}})
async def broken(ctx: HttpContext) -> None:
await ctx.send_str(200, 'x')
with self.assertRaises(ValueError) as caught:
generate_spec(app, title='Broken', version='1.0.0',
components={'schemas': {'User': {}}})
message = str(caught.exception)
self.assertIn('#/components/schemas/Missing', message)
self.assertIn('GET /broken', message)
def test_component_to_component_ref_validates(self) -> None:
app = KayaApp()
spec = generate_spec(app, title='Nested', version='1.0.0', components={'schemas': {
'User': {'type': 'object'},
'Page': {'type': 'object',
'properties': {'results': {'type': 'array', 'items': schema_ref('User')}}},
}})
self.assertIn('components', spec)
def test_external_refs_are_ignored(self) -> None:
app = KayaApp()
@app.GET('/external')
@operation(responses={200: {
'description': 'x',
'content': {'application/json': {'schema': {'$ref': './other.json#/Thing'}}},
}})
async def external(ctx: HttpContext) -> None:
await ctx.send_str(200, 'x')
spec = generate_spec(app, title='External', version='1.0.0')
self.assertNotIn('components', spec)
def test_generic_component_section_validated(self) -> None:
app = KayaApp()
@app.GET('/p')
@operation(parameters=[{'name': 'q', 'in': 'query',
'schema': {'$ref': '#/components/parameters/Missing'}}])
async def p(ctx: HttpContext) -> None:
await ctx.send_str(200, 'x')
with self.assertRaises(ValueError) as caught:
generate_spec(app, title='P', version='1.0.0', components={'parameters': {}})
self.assertIn('#/components/parameters/Missing', str(caught.exception))
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
+89
View File
@@ -0,0 +1,89 @@
# kaya-otel
OpenTelemetry tracing and metrics for the Kaya web framework.
Provides `OTelMixin`, a `KayaMixin` that instruments HTTP requests and
WebSocket connections with OpenTelemetry spans and exports HTTP server
metrics, shipping everything to an OTLP/HTTP collector.
## Usage
```python
from kaya.core import KayaApp, HttpContext
from kaya.otel import OTelMixin
app = KayaApp(mixins=[
OTelMixin(
service_name='my-service',
endpoint='http://localhost:4318',
)
])
@app.GET('/')
async def home(ctx: HttpContext):
await ctx.send_str(200, 'Hello World!')
```
## Parameters
- `service_name`: value of the `service.name` resource attribute.
- `endpoint`: base URL of an OTLP/HTTP collector; the signal paths
`/v1/traces` and `/v1/metrics` are appended. When `None`, the exporters
use their own defaults, including the standard
`OTEL_EXPORTER_OTLP_ENDPOINT` environment variable.
- `headers`: extra HTTP headers sent to the collector (e.g. authentication).
- `metric_export_interval_millis`: metric export interval
(default `60000`).
- `tracer_provider` / `meter_provider`: inject custom providers (e.g. with
in-memory exporters for tests) instead of the OTLP defaults. The mixin
only shuts down providers it created itself.
- `resource_attributes`: extra resource attributes merged with
`service.name` when the mixin creates the providers.
- `excluded_paths` / `excluded_path_regexes`: skip tracing and metrics for
matching paths (useful for health checks and metrics endpoints).
- `capture_request_headers` / `capture_response_headers`: opt-in HTTP header
capture as `http.request.header.<name>` / `http.response.header.<name>`
span attributes. Header names are normalized to lowercase with `-`
replaced by `_`; values are captured as string lists.
- `sanitize_headers`: headers captured as `REDACTED` (default:
`authorization`, `proxy-authorization`, `cookie`, `set-cookie`).
- `server_request_hook` / `server_response_hook` / `websocket_connect_hook` /
`websocket_close_hook`: optional synchronous callbacks invoked with the
span and the Kaya context/websocket at the corresponding lifecycle point.
Hook exceptions are logged and do not fail the request.
- `metrics_include_raw_path`: when `True`, metrics use raw `url.path`
attributes (legacy, potentially high cardinality). The default `False`
keeps metrics low-cardinality by using `http.route` when Kaya can resolve
a route template.
- `websocket_error_close_codes`: close codes that mark a websocket span as
failed. Defaults to protocol/application error codes such as `1002`,
`1003` and `1007`-`1011`.
## Behavior
- Every HTTP request gets a `SERVER` span named `<METHOD> <path>` with the
usual HTTP semantic attributes (`http.request.method`, `url.path`,
`client.address`, `server.address`, ...). When Kaya resolves a route
template, the span name is updated to `<METHOD> <route template>` and
`http.route` is set. The response status code is recorded as
`http.response.status_code` when the handler sends the response; 5xx
statuses mark the span as failed. Exceptions escaping the handler are
recorded on the span and also mark it as failed.
- Every WebSocket connection gets one span for its whole lifetime, carrying
the applicable standard attributes: `url.scheme`, `url.path`, `url.query`,
`network.protocol.name` (`websocket`), `network.transport`,
`client.address`/`client.port` and `server.address`/`server.port`. The
close code is recorded as `kaya.websocket.close_code`; exceptions and
configured error close codes mark the span as failed and set `error.type`
(the exception class name, or the close code).
- W3C `traceparent`/`tracestate` headers on incoming requests are honored,
so traces propagate from upstream services.
- Metrics:
- `http.server.request.duration` (histogram, seconds), with
`http.request.method`, `http.route` when known and
`http.response.status_code` attributes.
- `http.server.active_requests` (up-down counter), with
`http.request.method` attributes.
`OTelMixin` is a `KayaMixin`, so the app stays a `KayaApp` and both ASGI and
RSGI keep working.
+53
View File
@@ -0,0 +1,53 @@
[build-system]
requires = ["setuptools>=61.0", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"
[project]
name = "kaya-otel"
dynamic = ["version"]
authors = [
{ name="Walter Oggioni", email="oggioni.walter@gmail.com" },
]
description = "OpenTelemetry tracing and metrics for the Kaya lightweight ASGI web framework"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
classifiers = [
'Development Status :: 3 - Alpha',
'Topic :: Utilities',
'Intended Audience :: System Administrators',
'Intended Audience :: Developers',
'Environment :: Console',
'Programming Language :: Python :: 3',
]
dependencies = [
"kaya-core",
"opentelemetry-sdk",
"opentelemetry-exporter-otlp-proto-http",
]
[project.urls]
"Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
[tool.mypy]
python_version = "3.12"
disallow_untyped_defs = true
show_error_codes = true
no_implicit_optional = true
warn_return_any = true
warn_unused_ignores = true
exclude = ["scripts", "docs", "test"]
strict = true
[tool.setuptools_scm]
root = "../.."
version_file = "src/kaya/otel/_version.py"
[tool.setuptools_scm.tag]
prefix = "release/"
@@ -0,0 +1,10 @@
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
from ._mixin import OTelMixin
__all__ = [
'OTelMixin',
]
+518
View File
@@ -0,0 +1,518 @@
import re
import time
from asyncio import AbstractEventLoop
from logging import getLogger
from pathlib import Path
from typing import (
Any,
AsyncGenerator,
Callable,
Dict,
Mapping,
Optional,
Sequence,
Tuple,
)
from kaya.core import HttpContext, KayaApp, KayaMixin, WebSocket, WebSocketMessage
from kaya.core._types import StrOrStrings
from opentelemetry import context as otel_context
from opentelemetry import trace
from opentelemetry.context import Context
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.metrics import Histogram, UpDownCounter
from opentelemetry.propagators.textmap import Getter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.trace import Span, SpanKind, Status, StatusCode, Tracer
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
log = getLogger(__name__)
RequestHook = Callable[[Span, HttpContext], None]
WebSocketHook = Callable[[Span, WebSocket], None]
_DEFAULT_SANITIZE_HEADERS = (
'authorization',
'proxy-authorization',
'cookie',
'set-cookie',
)
_DEFAULT_WEBSOCKET_ERROR_CLOSE_CODES = frozenset((1002, 1003, 1007, 1008, 1009, 1010, 1011))
class _HeadersGetter(Getter[Mapping[str, Sequence[str]]]):
"""Extract propagation headers from a kaya request/websocket context.
Kaya header mappings have lowercase names with one or more values each,
which is all the W3C trace-context propagator needs.
"""
def get(self, carrier: Mapping[str, Sequence[str]], key: str) -> Optional[list[str]]:
values = carrier.get(key.lower())
if not values:
return None
return list(values)
def keys(self, carrier: Mapping[str, Sequence[str]]) -> list[str]:
return list(carrier.keys())
_PROPAGATOR = TraceContextTextMapPropagator()
_GETTER = _HeadersGetter()
def _server_attributes(ctx: HttpContext) -> Dict[str, Any]:
attributes: Dict[str, Any] = {
'http.request.method': str(ctx.method),
'url.scheme': ctx.scheme,
'url.path': ctx.path,
}
if ctx.query_string:
attributes['url.query'] = ctx.query_string
if ctx.client is not None:
attributes['client.address'] = ctx.client[0]
attributes['client.port'] = ctx.client[1]
if ctx.server is not None:
attributes['server.address'] = ctx.server[0]
if ctx.server[1] is not None:
attributes['server.port'] = ctx.server[1]
return attributes
def _websocket_attributes(ws: WebSocket) -> Dict[str, Any]:
"""Standard attributes applicable to a websocket connection span.
OpenTelemetry defines no websocket-specific conventions; these are the
general URL, network and peer attributes that apply to the connection.
"""
attributes: Dict[str, Any] = {
'url.scheme': ws.scheme,
'url.path': ws.path,
'network.protocol.name': 'websocket',
'network.transport': 'tcp',
}
if ws.query_string:
attributes['url.query'] = ws.query_string
if ws.client is not None:
attributes['client.address'] = ws.client[0]
attributes['client.port'] = ws.client[1]
if ws.server is not None:
attributes['server.address'] = ws.server[0]
if ws.server[1] is not None:
attributes['server.port'] = ws.server[1]
return attributes
def _normalize_header_name(name: str) -> str:
return name.lower().replace('-', '_')
def _header_values(value: StrOrStrings | Sequence[str]) -> list[str]:
if isinstance(value, str):
return [value]
return list(value)
class _SpanHolder:
"""Mutable per-request slot shared by the tracing hooks and wrappers."""
__slots__ = ('span', 'token', 'status_code', 'start_ns', 'active_attributes', 'close_code')
def __init__(self, span: Span, token: object, active_attributes: Dict[str, Any]) -> None:
self.span = span
self.token = token
self.status_code: Optional[int] = None
self.start_ns = time.perf_counter_ns()
self.active_attributes = active_attributes
self.close_code: Optional[int] = None
class _TracedHttpContext(HttpContext):
"""HttpContext wrapper that records the response status code on the span.
Attributes not explicitly overridden are delegated to the wrapped context
via ``__getattr__``, so it works with any concrete ``HttpContext`` (ASGI
or RSGI) and preserves attributes set by other mixins (e.g. ``session``).
"""
def __init__(self, ctx: HttpContext, holder: _SpanHolder, mixin: 'OTelMixin') -> None:
object.__setattr__(self, '_ctx', ctx)
object.__setattr__(self, 'session', ctx.session)
object.__setattr__(self, '_holder', holder)
object.__setattr__(self, '_mixin', mixin)
def __getattr__(self, name: str) -> Any:
if name == '_ctx':
raise AttributeError(name)
return getattr(self._ctx, name)
def _sent(self, status: int, headers: Optional[Mapping[str, StrOrStrings]]) -> None:
self._holder.status_code = status
self._holder.span.set_attribute('http.response.status_code', status)
self._mixin._capture_response_headers(self._holder.span, headers)
async def stream_body(self,
status: int,
body_generator: AsyncGenerator[bytes, None],
headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.stream_body(status, body_generator, headers)
async def send_bytes(self, status: int, body: bytes, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_bytes(status, body, headers)
async def send_str(self, status: int, body: str, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_str(status, body, headers)
async def send_file(self, status: int, path: Path, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_file(status, path, headers)
async def send_empty(self, status: int, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_empty(status, headers)
class _TracedWebSocket(WebSocket):
"""WebSocket wrapper that keeps the connection span reachable.
Everything is delegated to the wrapped socket via ``__getattr__``, so it
works with any concrete ``WebSocket`` (ASGI or RSGI) and preserves
attributes set by other mixins (e.g. ``session``).
"""
def __init__(self, ws: WebSocket, holder: _SpanHolder) -> None:
object.__setattr__(self, '_ws', ws)
object.__setattr__(self, 'session', ws.session)
object.__setattr__(self, '_holder', holder)
def __getattr__(self, name: str) -> Any:
if name == '_ws':
raise AttributeError(name)
return getattr(self._ws, name)
async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ws.accept(headers)
async def receive(self) -> WebSocketMessage:
return await self._ws.receive()
async def send_text(self, data: str) -> None:
await self._ws.send_text(data)
async def send_bytes(self, data: bytes) -> None:
await self._ws.send_bytes(data)
async def close(self, code: int = 1000) -> None:
self._holder.close_code = code
await self._ws.close(code)
async def __anext__(self) -> WebSocketMessage:
return await self._ws.__anext__()
class OTelMixin(KayaMixin):
"""Kaya mixin adding OpenTelemetry tracing and metrics.
HTTP requests get a ``SERVER`` span named ``<METHOD> <path>`` carrying the
usual HTTP semantic attributes; the response status code is recorded when
the handler sends the response and 5xx statuses mark the span as failed.
Exceptions escaping the handler are recorded and mark the span as failed.
WebSocket connections get one span for the whole connection lifetime,
carrying the applicable standard websocket URL, network and peer
attributes (``url.scheme``, ``url.path``, ``network.protocol.name``,
``client.address``/``server.address``, ...); exceptions and configured
error close codes set ``error.type`` and mark the span as failed.
W3C ``traceparent``/``tracestate`` headers on incoming requests are
honored, so traces propagate from upstream services.
Metrics exported on the meter provider:
- ``http.server.request.duration`` (histogram, seconds), with
``http.request.method``, ``http.route`` when known and
``http.response.status_code`` attributes;
- ``http.server.active_requests`` (up-down counter), with
``http.request.method`` attributes.
Example::
app = KayaApp(mixins=[
OTelMixin(
service_name='my-service',
endpoint='http://localhost:4318',
)
])
``endpoint`` is the base URL of an OTLP/HTTP collector (the signal paths
``/v1/traces`` and ``/v1/metrics`` are appended); when ``None`` the
exporters fall back to their own defaults, including the standard
``OTEL_EXPORTER_OTLP_ENDPOINT`` environment variable. Custom
``tracer_provider``/``meter_provider`` instances (e.g. with in-memory
exporters for tests) can be injected instead of the OTLP defaults; the
mixin only shuts down providers it created itself.
"""
def __init__(self,
service_name: str,
endpoint: Optional[str] = None,
headers: Optional[Mapping[str, str]] = None,
metric_export_interval_millis: float = 60000,
tracer_provider: Optional[TracerProvider] = None,
meter_provider: Optional[MeterProvider] = None,
resource_attributes: Optional[Mapping[str, Any]] = None,
excluded_paths: Sequence[str] = (),
excluded_path_regexes: Sequence[str] = (),
capture_request_headers: Sequence[str] = (),
capture_response_headers: Sequence[str] = (),
sanitize_headers: Sequence[str] = _DEFAULT_SANITIZE_HEADERS,
server_request_hook: Optional[RequestHook] = None,
server_response_hook: Optional[RequestHook] = None,
websocket_connect_hook: Optional[WebSocketHook] = None,
websocket_close_hook: Optional[WebSocketHook] = None,
metrics_include_raw_path: bool = False,
websocket_error_close_codes: Optional[Sequence[int]] = None) -> None:
self._owns_tracer_provider = tracer_provider is None
self._owns_meter_provider = meter_provider is None
if tracer_provider is None:
tracer_provider = TracerProvider(resource=self._create_resource(service_name, resource_attributes))
span_exporter = OTLPSpanExporter(
endpoint=f'{endpoint}/v1/traces' if endpoint else None,
headers=dict(headers) if headers else None,
)
tracer_provider.add_span_processor(BatchSpanProcessor(span_exporter))
if meter_provider is None:
meter_provider = MeterProvider(
resource=self._create_resource(service_name, resource_attributes),
metric_readers=[
PeriodicExportingMetricReader(
OTLPMetricExporter(
endpoint=f'{endpoint}/v1/metrics' if endpoint else None,
headers=dict(headers) if headers else None,
),
export_interval_millis=metric_export_interval_millis,
)
],
)
self._tracer_provider = tracer_provider
self._meter_provider = meter_provider
self._tracer: Tracer = tracer_provider.get_tracer('kaya.otel')
meter = meter_provider.get_meter('kaya.otel')
self._request_duration: Histogram = meter.create_histogram(
'http.server.request.duration',
unit='s',
description='Duration of HTTP server requests',
)
self._active_requests: UpDownCounter = meter.create_up_down_counter(
'http.server.active_requests',
unit='{request}',
description='Number of in-flight HTTP server requests',
)
self._app: Optional[KayaApp] = None
self._excluded_paths = frozenset(excluded_paths)
self._excluded_path_regexes = tuple(re.compile(regex) for regex in excluded_path_regexes)
self._request_headers_to_capture = tuple(capture_request_headers)
self._response_headers_to_capture = tuple(capture_response_headers)
self._sanitize_headers = frozenset(name.lower() for name in sanitize_headers)
self._server_request_hook = server_request_hook
self._server_response_hook = server_response_hook
self._websocket_connect_hook = websocket_connect_hook
self._websocket_close_hook = websocket_close_hook
self._metrics_include_raw_path = metrics_include_raw_path
self._websocket_error_close_codes = (
frozenset(websocket_error_close_codes)
if websocket_error_close_codes is not None
else _DEFAULT_WEBSOCKET_ERROR_CLOSE_CODES
)
# Live spans keyed by the id of the (wrapped) context/websocket the
# after hooks receive; entries are removed when the span ends.
self._spans: Dict[int, _SpanHolder] = {}
@staticmethod
def _create_resource(service_name: str,
resource_attributes: Optional[Mapping[str, Any]]) -> Resource:
attributes: Dict[str, Any] = {'service.name': service_name}
if resource_attributes:
attributes.update(resource_attributes)
return Resource.create(attributes)
def apply(self, app: KayaApp) -> None:
self._app = app
app.add_before_request_hook(self._before_request)
app.add_after_request_hook(self._after_request)
app.add_before_websocket_hook(self._before_websocket)
app.add_after_websocket_hook(self._after_websocket)
def _is_excluded(self, path: str) -> bool:
if path in self._excluded_paths:
return True
return any(regex.search(path) is not None for regex in self._excluded_path_regexes)
@staticmethod
def _call_hook(hook: Optional[Callable[[Span, Any], None]], span: Span, value: Any) -> None:
if hook is None:
return
try:
hook(span, value)
except Exception:
log.exception('OpenTelemetry span hook failed')
def _capture_headers(self,
span: Span,
headers: Mapping[str, StrOrStrings | Sequence[str]],
configured: Sequence[str],
prefix: str) -> None:
if not configured:
return
lower_headers = {name.lower(): value for name, value in headers.items()}
for name in configured:
key = name.lower()
if key not in lower_headers:
continue
values = _header_values(lower_headers[key])
if key in self._sanitize_headers:
values = ['REDACTED'] * len(values)
span.set_attribute(f'{prefix}.{_normalize_header_name(name)}', values)
def _capture_request_headers(self, span: Span, headers: Mapping[str, Sequence[str]]) -> None:
self._capture_headers(span, headers, self._request_headers_to_capture, 'http.request.header')
def _capture_response_headers(self,
span: Span,
headers: Optional[Mapping[str, StrOrStrings]]) -> None:
if headers is None:
return
self._capture_headers(span, headers, self._response_headers_to_capture, 'http.response.header')
def _start_span(self, name: str, attributes: Mapping[str, Any],
headers: Mapping[str, Sequence[str]]) -> _SpanHolder:
parent: Context = _PROPAGATOR.extract(carrier=headers, getter=_GETTER)
span = self._tracer.start_span(
name,
context=parent,
kind=SpanKind.SERVER,
attributes=dict(attributes),
)
self._capture_request_headers(span, headers)
token = otel_context.attach(trace.set_span_in_context(span, parent))
return _SpanHolder(span, token, {})
@staticmethod
def _finish_span(holder: _SpanHolder, error: bool) -> None:
if error:
holder.span.set_status(Status(StatusCode.ERROR))
holder.span.end()
otel_context.detach(holder.token) # type: ignore[arg-type]
def _pop_holder(self, obj: Any) -> Optional[_SpanHolder]:
"""Find and remove the span holder for a context/websocket.
The before hooks key holders by the id of the context they received,
but after hooks run on the outermost wrapper (later mixins may have
wrapped the context again), so the id may not match. In that case the
holder is reachable through the wrapper delegation chain as
``_holder`` and the stale dict entry is reaped by identity.
"""
holder = self._spans.pop(id(obj), None)
if holder is not None:
return holder
candidate = getattr(obj, '_holder', None)
if not isinstance(candidate, _SpanHolder):
return None
for key, value in list(self._spans.items()):
if value is candidate:
del self._spans[key]
return candidate
def _route_template(self, ctx: HttpContext) -> Optional[str]:
if self._app is None:
return None
return self._app.route_template(ctx.path, ctx.method)
async def _before_request(self, ctx: HttpContext) -> Optional[HttpContext]:
if self._is_excluded(ctx.path):
return None
holder = self._start_span(
f'{ctx.method} {ctx.path}',
_server_attributes(ctx),
ctx.headers,
)
holder.active_attributes = {'http.request.method': str(ctx.method)}
if self._metrics_include_raw_path:
holder.active_attributes['url.path'] = ctx.path
self._spans[id(ctx)] = holder
self._active_requests.add(1, holder.active_attributes)
self._call_hook(self._server_request_hook, holder.span, ctx)
return _TracedHttpContext(ctx, holder, self)
async def _after_request(self, ctx: HttpContext) -> None:
holder = self._pop_holder(ctx)
if holder is None:
return
status_code = holder.status_code
exception = getattr(ctx, 'exception', None)
error = status_code is not None and status_code >= 500
if isinstance(exception, BaseException):
holder.span.record_exception(exception)
error = True
route_template = self._route_template(ctx)
if route_template is not None:
holder.span.update_name(f'{ctx.method} {route_template}')
holder.span.set_attribute('http.route', route_template)
self._call_hook(self._server_response_hook, holder.span, ctx)
self._finish_span(holder, error)
self._active_requests.add(-1, holder.active_attributes)
metric_attributes: Dict[str, Any] = {
'http.request.method': str(ctx.method),
}
if self._metrics_include_raw_path:
metric_attributes['url.path'] = ctx.path
elif route_template is not None:
metric_attributes['http.route'] = route_template
if status_code is not None:
metric_attributes['http.response.status_code'] = status_code
duration = (time.perf_counter_ns() - holder.start_ns) / 1e9
self._request_duration.record(duration, metric_attributes)
async def _before_websocket(self, ws: WebSocket) -> Optional[WebSocket]:
if self._is_excluded(ws.path):
return None
holder = self._start_span(f'WS {ws.path}', _websocket_attributes(ws), ws.headers)
self._spans[id(ws)] = holder
self._call_hook(self._websocket_connect_hook, holder.span, ws)
return _TracedWebSocket(ws, holder)
async def _after_websocket(self, ws: WebSocket) -> None:
holder = self._pop_holder(ws)
if holder is None:
return
exception = getattr(ws, 'exception', None)
error = isinstance(exception, BaseException)
if isinstance(exception, BaseException):
holder.span.record_exception(exception)
holder.span.set_attribute('error.type', type(exception).__qualname__)
close_code = holder.close_code
if close_code is not None:
holder.span.set_attribute('kaya.websocket.close_code', close_code)
if close_code in self._websocket_error_close_codes:
error = True
if not isinstance(exception, BaseException):
# Mirror the HTTP conventions, where server-side error
# statuses are recorded as ``error.type`` values.
holder.span.set_attribute('error.type', str(close_code))
self._call_hook(self._websocket_close_hook, holder.span, ws)
self._finish_span(holder, error)
def shutdown(self, loop: AbstractEventLoop) -> None:
# Only providers created by this mixin are shut down; injected ones
# stay under the caller's control.
if self._owns_tracer_provider:
self._tracer_provider.shutdown()
if self._owns_meter_provider:
self._meter_provider.shutdown()
Whitespace-only changes.
+420
View File
@@ -0,0 +1,420 @@
import json
import unittest
from typing import Any, Optional, Tuple
import httpx
from pwo import async_test
from kaya.core import HttpContext, KayaApp
from kaya.core._asgi import AsgiWebSocket
from kaya.otel import OTelMixin
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import InMemoryMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import StatusCode
TRACEPARENT = '00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01'
def make_mixin(**kwargs) -> Tuple[OTelMixin, InMemorySpanExporter, InMemoryMetricReader]:
resource = Resource.create({'service.name': 'test-service'})
span_exporter = InMemorySpanExporter()
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter))
metric_reader = InMemoryMetricReader()
meter_provider = MeterProvider(resource=resource, metric_readers=[metric_reader])
mixin = OTelMixin(
service_name='test-service',
tracer_provider=tracer_provider,
meter_provider=meter_provider,
**kwargs,
)
return mixin, span_exporter, metric_reader
def make_app(mixin: OTelMixin) -> KayaApp:
app = KayaApp(mixins=[mixin])
@app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
await ctx.send_str(200, json.dumps({'ok': True}))
@app.GET('/boom')
async def boom(ctx: HttpContext) -> None:
await ctx.send_str(500, 'boom')
return app
async def request(app: KayaApp, path: str = '/hello', headers: Optional[dict[str, str]] = None) -> httpx.Response:
transport = httpx.ASGITransport(app=app, client=('127.0.0.1', 123))
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as http_client:
return await http_client.get(path, headers=headers)
class HttpTracingTest(unittest.TestCase):
@async_test
async def test_request_produces_server_span(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
r = await request(app)
self.assertEqual(200, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual('GET /hello', span.name)
assert span.attributes is not None
self.assertEqual('GET', span.attributes['http.request.method'])
self.assertEqual('/hello', span.attributes['url.path'])
self.assertEqual(200, span.attributes['http.response.status_code'])
self.assertEqual(StatusCode.UNSET, span.status.status_code)
@async_test
async def test_5xx_marks_span_as_error(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
r = await request(app, '/boom')
self.assertEqual(500, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(500, spans[0].attributes['http.response.status_code']) # type: ignore[index]
self.assertEqual(StatusCode.ERROR, spans[0].status.status_code)
@async_test
async def test_traceparent_header_propagates(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
await request(app, headers={'traceparent': TRACEPARENT})
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual(0x4bf92f3577b34da6a3ce929d0e0e4736, span.context.trace_id) # type: ignore[union-attr]
assert span.parent is not None
self.assertEqual(0x00f067aa0ba902b7, span.parent.span_id)
@async_test
async def test_unmatched_route_still_traced(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
r = await request(app, '/nowhere')
self.assertEqual(404, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(404, spans[0].attributes['http.response.status_code']) # type: ignore[index]
@async_test
async def test_no_leaked_spans_after_requests(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
await request(app)
await request(app, '/boom')
self.assertEqual(0, len(mixin._spans))
self.assertEqual(2, len(span_exporter.get_finished_spans()))
@async_test
async def test_exception_marks_span_as_error(self):
mixin, span_exporter, _ = make_mixin()
app = KayaApp(mixins=[mixin])
@app.GET('/raises')
async def raises(ctx: HttpContext) -> None:
raise RuntimeError('boom')
transport = httpx.ASGITransport(app=app, client=('127.0.0.1', 123))
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as http_client:
with self.assertRaises(RuntimeError):
await http_client.get('/raises')
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual(StatusCode.ERROR, span.status.status_code)
self.assertTrue(any(event.name == 'exception' for event in span.events))
self.assertEqual(0, len(mixin._spans))
@async_test
async def test_excluded_path_skips_tracing_and_metrics(self):
mixin, span_exporter, metric_reader = make_mixin(excluded_paths=('/health',))
app = KayaApp(mixins=[mixin])
@app.GET('/health')
async def health(ctx: HttpContext) -> None:
await ctx.send_str(200, 'ok')
@app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
await ctx.send_str(200, 'hello')
await request(app, '/health')
await request(app, '/hello')
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual('GET /hello', spans[0].name)
metrics = MetricsTest._metric_names(metric_reader)
duration_points = list(metrics['http.server.request.duration'].data.data_points)
self.assertEqual(1, len(duration_points))
self.assertEqual(1, duration_points[0].count)
@async_test
async def test_route_template_used_for_span_name_and_metrics(self):
mixin, span_exporter, metric_reader = make_mixin()
app = KayaApp(mixins=[mixin])
@app.GET('/items/${item_id:int}')
async def item(ctx: HttpContext, item_id: int) -> None:
await ctx.send_str(200, str(item_id))
r = await request(app, '/items/123')
self.assertEqual(200, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual('GET /items/${item_id:int}', span.name)
assert span.attributes is not None
self.assertEqual('/items/${item_id:int}', span.attributes['http.route'])
self.assertEqual('/items/123', span.attributes['url.path'])
metrics = MetricsTest._metric_names(metric_reader)
duration_points = list(metrics['http.server.request.duration'].data.data_points)
self.assertEqual(1, len(duration_points))
duration_attributes = dict(duration_points[0].attributes)
self.assertEqual('GET', duration_attributes['http.request.method'])
self.assertEqual('/items/${item_id:int}', duration_attributes['http.route'])
self.assertEqual(200, duration_attributes['http.response.status_code'])
self.assertNotIn('url.path', duration_attributes)
active_points = list(metrics['http.server.active_requests'].data.data_points)
self.assertEqual(1, len(active_points))
self.assertEqual({'http.request.method': 'GET'}, dict(active_points[0].attributes))
@async_test
async def test_header_capture_and_sanitization(self):
mixin, span_exporter, _ = make_mixin(
capture_request_headers=('X-Request-Id', 'Authorization'),
capture_response_headers=('X-Response-Id', 'Set-Cookie'),
)
app = KayaApp(mixins=[mixin])
@app.GET('/headers')
async def headers(ctx: HttpContext) -> None:
await ctx.send_str(200, 'ok', {
'X-Response-Id': 'res-1',
'Set-Cookie': 'sid=secret',
})
r = await request(app, '/headers', headers={
'X-Request-Id': 'req-1',
'Authorization': 'Bearer secret',
})
self.assertEqual(200, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
attributes = spans[0].attributes
assert attributes is not None
self.assertEqual(['req-1'], list(attributes['http.request.header.x_request_id']))
self.assertEqual(['REDACTED'], list(attributes['http.request.header.authorization']))
self.assertEqual(['res-1'], list(attributes['http.response.header.x_response_id']))
self.assertEqual(['REDACTED'], list(attributes['http.response.header.set_cookie']))
@async_test
async def test_http_hooks_are_called(self):
calls = []
def request_hook(span: Any, ctx: HttpContext) -> None:
calls.append(('request', ctx.path))
span.set_attribute('test.request_hook', True)
def response_hook(span: Any, ctx: HttpContext) -> None:
calls.append(('response', ctx.path))
span.set_attribute('test.response_hook', True)
mixin, span_exporter, _ = make_mixin(
server_request_hook=request_hook,
server_response_hook=response_hook,
)
app = make_app(mixin)
await request(app)
self.assertEqual([('request', '/hello'), ('response', '/hello')], calls)
attributes = span_exporter.get_finished_spans()[0].attributes
assert attributes is not None
self.assertTrue(attributes['test.request_hook'])
self.assertTrue(attributes['test.response_hook'])
class MetricsTest(unittest.TestCase):
@staticmethod
def _metric_names(metric_reader: InMemoryMetricReader) -> dict:
data = metric_reader.get_metrics_data()
return {m.name: m for rm in data.resource_metrics for m in rm.scope_metrics[0].metrics
if rm.scope_metrics}
@async_test
async def test_duration_histogram_and_active_requests(self):
mixin, _, metric_reader = make_mixin()
app = make_app(mixin)
await request(app)
metrics = self._metric_names(metric_reader)
self.assertIn('http.server.request.duration', metrics)
self.assertIn('http.server.active_requests', metrics)
duration_points = list(metrics['http.server.request.duration'].data.data_points)
self.assertEqual(1, len(duration_points))
self.assertEqual(1, duration_points[0].count)
self.assertGreaterEqual(duration_points[0].sum, 0)
active_points = list(metrics['http.server.active_requests'].data.data_points)
self.assertEqual(1, len(active_points))
self.assertEqual(0, active_points[0].value)
class WebSocketTracingTest(unittest.TestCase):
@staticmethod
def _make_ws(headers=(), query_string: bytes = b''):
async def send(message):
pass
async def receive():
return {'type': 'websocket.connect'}
scope = {
'type': 'websocket',
'path': '/ws/games/abc',
'query_string': query_string,
'scheme': 'ws',
'client': ('127.0.0.1', 12345),
'server': ('127.0.0.1', 80),
'headers': headers,
}
return AsgiWebSocket(scope, receive, send)
@async_test
async def test_websocket_lifecycle_produces_span(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
self.assertIsNotNone(wrapped)
self.assertEqual(1, len(mixin._spans))
await mixin._after_websocket(wrapped) # type: ignore[arg-type]
self.assertEqual(0, len(mixin._spans))
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual('WS /ws/games/abc', span.name)
assert span.attributes is not None
self.assertEqual('ws', span.attributes['url.scheme'])
self.assertEqual('/ws/games/abc', span.attributes['url.path'])
self.assertNotIn('url.query', span.attributes)
self.assertEqual('websocket', span.attributes['network.protocol.name'])
self.assertEqual('tcp', span.attributes['network.transport'])
self.assertEqual('127.0.0.1', span.attributes['client.address'])
self.assertEqual(12345, span.attributes['client.port'])
self.assertEqual('127.0.0.1', span.attributes['server.address'])
self.assertEqual(80, span.attributes['server.port'])
@async_test
async def test_websocket_query_string_recorded(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws(query_string=b'room=1&token=abc')
wrapped = await mixin._before_websocket(ws)
await mixin._after_websocket(wrapped) # type: ignore[arg-type]
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
assert spans[0].attributes is not None
self.assertEqual('room=1&token=abc', spans[0].attributes['url.query'])
@async_test
async def test_websocket_traceparent_propagates(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws(headers=[(b'traceparent', TRACEPARENT.encode())])
wrapped = await mixin._before_websocket(ws)
await mixin._after_websocket(wrapped) # type: ignore[arg-type]
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(0x4bf92f3577b34da6a3ce929d0e0e4736, spans[0].context.trace_id) # type: ignore[union-attr]
@async_test
async def test_wrapped_websocket_delegates(self):
mixin, _, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
self.assertEqual('/ws/games/abc', wrapped.path)
self.assertEqual(('127.0.0.1', 12345), wrapped.client)
await mixin._after_websocket(wrapped)
@async_test
async def test_websocket_error_close_code_marks_span_error(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
await wrapped.close(1011)
await mixin._after_websocket(wrapped)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(StatusCode.ERROR, spans[0].status.status_code)
assert spans[0].attributes is not None
self.assertEqual(1011, spans[0].attributes['kaya.websocket.close_code'])
self.assertEqual('1011', spans[0].attributes['error.type'])
@async_test
async def test_websocket_exception_marks_span_error(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
wrapped.exception = RuntimeError('ws boom')
await mixin._after_websocket(wrapped)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(StatusCode.ERROR, spans[0].status.status_code)
self.assertTrue(any(event.name == 'exception' for event in spans[0].events))
assert spans[0].attributes is not None
self.assertEqual('RuntimeError', spans[0].attributes['error.type'])
@async_test
async def test_websocket_hooks_are_called(self):
calls = []
def connect_hook(span: Any, ws: Any) -> None:
calls.append(('connect', ws.path))
span.set_attribute('test.websocket_connect_hook', True)
def close_hook(span: Any, ws: Any) -> None:
calls.append(('close', ws.path))
span.set_attribute('test.websocket_close_hook', True)
mixin, span_exporter, _ = make_mixin(
websocket_connect_hook=connect_hook,
websocket_close_hook=close_hook,
)
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
await wrapped.close(1000)
await mixin._after_websocket(wrapped)
self.assertEqual([('connect', '/ws/games/abc'), ('close', '/ws/games/abc')], calls)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(StatusCode.UNSET, spans[0].status.status_code)
assert spans[0].attributes is not None
self.assertNotIn('error.type', spans[0].attributes)
self.assertTrue(spans[0].attributes['test.websocket_connect_hook'])
self.assertTrue(spans[0].attributes['test.websocket_close_hook'])
if __name__ == '__main__':
unittest.main()
-5
View File
@@ -27,11 +27,6 @@ dependencies = [
"pwo", "pwo",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
+48 -6
View File
@@ -17,13 +17,21 @@ from typing import (
from granian._granian import ( # type: ignore[attr-defined] from granian._granian import ( # type: ignore[attr-defined]
RSGIHTTPProtocol, RSGIHTTPProtocol,
RSGIHTTPScope, RSGIHTTPScope,
RSGIProtocolClosed,
RSGIWebsocketProtocol, RSGIWebsocketProtocol,
RSGIWebsocketScope, RSGIWebsocketScope,
RSGIWebsocketTransport, RSGIWebsocketTransport,
) )
from pwo import Maybe from pwo import Maybe
from kaya.core import AbstractKayaApp, HttpContext, HttpMethod, WebSocket, WebSocketMessage from kaya.core import (
AbstractKayaApp,
HttpContext,
HttpMethod,
WebSocket,
WebSocketDisconnect,
WebSocketMessage,
)
from kaya.core._types import StrOrStrings from kaya.core._types import StrOrStrings
@@ -109,6 +117,7 @@ class RsgiContext(HttpContext):
class RsgiWebSocket(WebSocket): class RsgiWebSocket(WebSocket):
_protocol: RSGIWebsocketProtocol _protocol: RSGIWebsocketProtocol
_transport: Optional[RSGIWebsocketTransport] _transport: Optional[RSGIWebsocketTransport]
_closed: bool
scheme: str scheme: str
path: str path: str
query_string: str query_string: str
@@ -124,6 +133,7 @@ class RsgiWebSocket(WebSocket):
) )
self._protocol = protocol self._protocol = protocol
self._transport = None self._transport = None
self._closed = False
self.scheme = scope.scheme self.scheme = scope.scheme
self.path = scope.path self.path = scope.path
self.query_string = scope.query_string self.query_string = scope.query_string
@@ -144,13 +154,28 @@ class RsgiWebSocket(WebSocket):
async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None: async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
# RSGI's websocket accept() takes no arguments (https://github.com/emmett-framework/granian/blob/master/docs/spec/RSGI.md), # RSGI's websocket accept() takes no arguments (https://github.com/emmett-framework/granian/blob/master/docs/spec/RSGI.md),
# so handshake headers (e.g. Set-Cookie) cannot be sent on RSGI; they are silently ignored here. # so handshake headers (e.g. Set-Cookie) cannot be sent on RSGI; they are silently ignored here.
self._transport = await self._protocol.accept() try:
self._transport = await self._protocol.accept()
except RSGIProtocolClosed as exc:
self._closed = True
raise WebSocketDisconnect(code=1006) from exc
async def receive(self) -> WebSocketMessage: async def receive(self) -> WebSocketMessage:
if self._transport is None: if self._transport is None:
raise RuntimeError('WebSocket connection has not been accepted yet') raise RuntimeError('WebSocket connection has not been accepted yet')
message = await self._transport.receive() if self._closed:
return WebSocketMessage(kind='close')
try:
message = await self._transport.receive()
except RSGIProtocolClosed:
# Granian raises this when the peer is gone without a close
# handshake (browser tab closed, network dropped, ...). Surface it
# as a close message so receive loops terminate normally instead
# of seeing an exception.
self._closed = True
return WebSocketMessage(kind='close')
if message.kind == 0: if message.kind == 0:
self._closed = True
return WebSocketMessage(kind='close') return WebSocketMessage(kind='close')
elif message.kind == 1: elif message.kind == 1:
return WebSocketMessage(kind='binary', data=message.data) return WebSocketMessage(kind='binary', data=message.data)
@@ -162,15 +187,32 @@ class RsgiWebSocket(WebSocket):
async def send_text(self, data: str) -> None: async def send_text(self, data: str) -> None:
if self._transport is None: if self._transport is None:
raise RuntimeError('WebSocket connection has not been accepted yet') raise RuntimeError('WebSocket connection has not been accepted yet')
await self._transport.send_str(data) if self._closed:
raise WebSocketDisconnect(code=1006)
try:
await self._transport.send_str(data)
except RSGIProtocolClosed as exc:
self._closed = True
raise WebSocketDisconnect(code=1006) from exc
async def send_bytes(self, data: bytes) -> None: async def send_bytes(self, data: bytes) -> None:
if self._transport is None: if self._transport is None:
raise RuntimeError('WebSocket connection has not been accepted yet') raise RuntimeError('WebSocket connection has not been accepted yet')
await self._transport.send_bytes(data) if self._closed:
raise WebSocketDisconnect(code=1006)
try:
await self._transport.send_bytes(data)
except RSGIProtocolClosed as exc:
self._closed = True
raise WebSocketDisconnect(code=1006) from exc
async def close(self, code: int = 1000) -> None: async def close(self, code: int = 1000) -> None:
self._protocol.close(code) if self._closed:
return
try:
self._protocol.close(code)
except RSGIProtocolClosed:
self._closed = True
async def __anext__(self) -> WebSocketMessage: async def __anext__(self) -> WebSocketMessage:
message = await self.receive() message = await self.receive()
+138 -10
View File
@@ -1,22 +1,150 @@
import unittest import unittest
from types import SimpleNamespace
from typing import Any
from granian.rsgi import ProtocolClosed
from pwo import async_test
from kaya.core import WebSocketDisconnect
from kaya.rsgi import RsgiWebSocket from kaya.rsgi import RsgiWebSocket
class FakeScope:
scheme = 'ws'
path = '/ws'
query_string = ''
headers: dict[str, list[str]] = {}
client = '127.0.0.1:12345'
server = '127.0.0.1:80'
class FakeTransport:
"""A transport whose peer vanished without a close handshake."""
def __init__(self) -> None:
self.receive_calls = 0
self.sent: list[Any] = []
async def receive(self) -> Any:
self.receive_calls += 1
raise ProtocolClosed()
async def send_str(self, data: str) -> None:
self.sent.append(data)
raise ProtocolClosed()
async def send_bytes(self, data: bytes) -> None:
self.sent.append(data)
raise ProtocolClosed()
class CleanCloseTransport(FakeTransport):
"""A transport that reports a close message (kind 0)."""
async def receive(self) -> Any:
self.receive_calls += 1
return SimpleNamespace(kind=0, data=None)
class FakeProtocol:
def __init__(self, transport: FakeTransport) -> None:
self._transport = transport
self.closed: list[int] = []
async def accept(self) -> FakeTransport:
return self._transport
def close(self, code: int) -> None:
self.closed.append(code)
class ClosedOnAcceptProtocol(FakeProtocol):
"""A protocol whose peer vanished during the handshake."""
async def accept(self) -> FakeTransport:
raise ProtocolClosed()
class RsgiWebSocketTest(unittest.TestCase): class RsgiWebSocketTest(unittest.TestCase):
def test_misconfigured_granian(self): def test_misconfigured_granian(self):
class FakeScope: class FakeProtocolWithoutWebsockets:
scheme = 'ws'
path = '/ws'
query_string = ''
headers = {}
client = '127.0.0.1:12345'
server = '127.0.0.1:80'
class FakeProtocol:
pass pass
with self.assertRaises(RuntimeError) as ctx: with self.assertRaises(RuntimeError) as ctx:
RsgiWebSocket(FakeScope(), FakeProtocol()) # type: ignore[arg-type] RsgiWebSocket(FakeScope(), FakeProtocolWithoutWebsockets()) # type: ignore[arg-type]
self.assertIn('Granian was not configured for websockets', str(ctx.exception)) self.assertIn('Granian was not configured for websockets', str(ctx.exception))
async def _socket(self, transport: FakeTransport) -> tuple[RsgiWebSocket, FakeProtocol]:
protocol = FakeProtocol(transport)
ws = RsgiWebSocket(FakeScope(), protocol) # type: ignore[arg-type]
await ws.accept()
return ws, protocol
@async_test
async def test_receive_returns_close_when_transport_is_gone(self):
ws, _ = await self._socket(FakeTransport())
message = await ws.receive()
self.assertEqual('close', message.kind)
@async_test
async def test_iteration_ends_when_transport_is_gone(self):
ws, _ = await self._socket(FakeTransport())
messages = [message async for message in ws]
self.assertEqual([], messages)
@async_test
async def test_receive_remembers_the_close(self):
transport = FakeTransport()
ws, _ = await self._socket(transport)
self.assertEqual('close', (await ws.receive()).kind)
self.assertEqual('close', (await ws.receive()).kind)
self.assertEqual(1, transport.receive_calls)
@async_test
async def test_send_text_raises_disconnect_when_transport_is_gone(self):
ws, _ = await self._socket(FakeTransport())
with self.assertRaises(WebSocketDisconnect) as ctx:
await ws.send_text('hello')
self.assertEqual(1006, ctx.exception.code)
@async_test
async def test_send_bytes_raises_disconnect_when_transport_is_gone(self):
ws, _ = await self._socket(FakeTransport())
with self.assertRaises(WebSocketDisconnect) as ctx:
await ws.send_bytes(b'hello')
self.assertEqual(1006, ctx.exception.code)
@async_test
async def test_sends_after_disconnect_do_not_touch_the_transport(self):
transport = FakeTransport()
ws, _ = await self._socket(transport)
with self.assertRaises(WebSocketDisconnect):
await ws.send_text('first')
with self.assertRaises(WebSocketDisconnect):
await ws.send_text('second')
self.assertEqual(['first'], transport.sent)
@async_test
async def test_send_after_a_clean_close_raises_disconnect(self):
transport = CleanCloseTransport()
ws, _ = await self._socket(transport)
self.assertEqual('close', (await ws.receive()).kind)
with self.assertRaises(WebSocketDisconnect):
await ws.send_text('hello')
self.assertEqual([], transport.sent)
@async_test
async def test_accept_raises_disconnect_when_transport_is_gone(self):
ws = RsgiWebSocket(FakeScope(), ClosedOnAcceptProtocol(FakeTransport())) # type: ignore[arg-type]
with self.assertRaises(WebSocketDisconnect) as ctx:
await ws.accept()
self.assertEqual(1006, ctx.exception.code)
@async_test
async def test_close_is_noop_once_the_transport_is_gone(self):
ws, protocol = await self._socket(FakeTransport())
await ws.receive()
await ws.close(1000)
self.assertEqual([], protocol.closed)
+16 -2
View File
@@ -32,11 +32,25 @@ and slides on each access when the session mixin passes a `max_age`. TTLs
larger than 30 days are automatically converted to absolute Unix timestamps, larger than 30 days are automatically converted to absolute Unix timestamps,
as required by the memcached protocol. as required by the memcached protocol.
## Session indexes
`MemcacheSessionStore` supports `Session.add_index()`,
`SessionStore.find_sessions()`, and `delete_by_index()`. Reverse memberships
are stored as sets and updated with memcached CAS operations. Bucket keys use
a SHA-256 digest of the index name and value, and stale members are pruned
during lookups.
Memcached's item-size limit (normally 1 MB) applies to each index bucket. Use
the Redis store when one index value can have a very large number of active
sessions.
## Serialization ## Serialization
Session data is serialized with `pickle` by default, so arbitrary Python Session data is serialized with `pickle` by default, so arbitrary Python
objects can be stored. A different serializer can be plugged in via the objects can be stored. The serialized value is an envelope containing the
`dumps`/`loads` arguments: session dictionary and index metadata; legacy plain-dict values are still
accepted. A different serializer can be plugged in via the `dumps`/`loads`
arguments:
```python ```python
import json import json
@@ -26,11 +26,6 @@ dependencies = [
"aiomcache>=0.8", "aiomcache>=0.8",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
@@ -1,12 +1,15 @@
import hashlib
import pickle import pickle
from time import time from time import time
from typing import Any, Callable, Optional from typing import Any, Callable, Collection, Optional, Sequence
import aiomcache import aiomcache
from kaya.session import Session, SessionStore from kaya.session import IndexKey, Session, SessionStore
_THIRTY_DAYS = 30 * 24 * 60 * 60 _THIRTY_DAYS = 30 * 24 * 60 * 60
_INDEX_GRACE_PERIOD = 60
_CAS_RETRIES = 5
def _exptime(max_age: int, clock: Callable[[], float]) -> int: def _exptime(max_age: int, clock: Callable[[], float]) -> int:
@@ -36,8 +39,14 @@ class MemcacheSessionStore(SessionStore):
Python objects can be stored. Custom serializers can be plugged in via Python objects can be stored. Custom serializers can be plugged in via
the ``dumps``/``loads`` arguments. Only connect this store to a trusted the ``dumps``/``loads`` arguments. Only connect this store to a trusted
memcached server, as pickle deserialization of untrusted data is unsafe. memcached server, as pickle deserialization of untrusted data is unsafe.
Reverse index buckets are pickled sets updated with memcached CAS
operations. Memcached's item-size limit applies to each bucket; Redis is a
better fit when one index value can have a very large number of sessions.
""" """
supports_indexes = True
def __init__( def __init__(
self, self,
client: aiomcache.Client, client: aiomcache.Client,
@@ -55,6 +64,69 @@ class MemcacheSessionStore(SessionStore):
def _key(self, session_id: str) -> bytes: def _key(self, session_id: str) -> bytes:
return f"{self._prefix}{session_id}".encode() return f"{self._prefix}{session_id}".encode()
def _index_key(self, name: str, value: str) -> bytes:
digest = hashlib.sha256(f"{name}\0{value}".encode()).hexdigest()
return f"{self._prefix}idx:{digest}".encode()
def _index_exptime(self, max_age: Optional[int]) -> int:
if max_age is None:
return 0
return _exptime(max_age + _INDEX_GRACE_PERIOD, self._clock)
@staticmethod
def _decode_bucket(payload: bytes) -> tuple[set[str], Optional[int]]:
members, max_age = pickle.loads(payload)
return set(members), max_age
async def _update_bucket(
self,
index_key: bytes,
*,
add: Collection[str] = (),
remove: Collection[str] = (),
max_age: Optional[int] = None,
) -> None:
for _ in range(_CAS_RETRIES):
payload, cas_token = await self._client.gets(index_key)
if cas_token is None:
if not add:
return
members = set(add)
created = await self._client.add(
index_key,
pickle.dumps((members, max_age)),
exptime=self._index_exptime(max_age),
)
if created:
return
continue
assert payload is not None
members, stored_max_age = self._decode_bucket(payload)
updated = (members | set(add)) - set(remove)
if updated == members:
if not add:
return
if await self._client.touch(
index_key,
self._index_exptime(max_age),
):
return
# The bucket may have expired between gets() and touch().
# Retry so it can be recreated instead of losing the index.
continue
effective_max_age = max_age if add else stored_max_age
if await self._client.cas(
index_key,
pickle.dumps((updated, effective_max_age)),
cas_token,
exptime=self._index_exptime(effective_max_age),
):
return
raise RuntimeError('Could not update session index after repeated CAS conflicts')
async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]: async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]:
key = self._key(session_id) key = self._key(session_id)
payload = await self._client.get(key) payload = await self._client.get(key)
@@ -65,12 +137,63 @@ class MemcacheSessionStore(SessionStore):
await self._client.touch(key, _exptime(max_age, self._clock)) await self._client.touch(key, _exptime(max_age, self._clock))
data = self._loads(payload) data = self._loads(payload)
return Session(session_id, data) return Session.from_payload(session_id, data)
async def save(self, session_id: str, session: Session, max_age: Optional[int] = None) -> None: async def save(self, session_id: str, session: Session, max_age: Optional[int] = None) -> None:
payload = self._dumps(dict(session)) # Index first: if the session write subsequently fails, a stale index
# entry is harmless and will be pruned, while a live unindexed session
# would be invisible to index-based invalidation.
for name, value in session.indexes:
await self._update_bucket(
self._index_key(name, value),
add={session_id},
max_age=max_age,
)
for name, value in session.removed_indexes:
await self._update_bucket(
self._index_key(name, value),
remove={session_id},
)
payload = self._dumps(session.to_payload())
exptime = _exptime(max_age, self._clock) if max_age is not None else 0 exptime = _exptime(max_age, self._clock) if max_age is not None else 0
await self._client.set(self._key(session_id), payload, exptime=exptime) await self._client.set(self._key(session_id), payload, exptime=exptime)
async def delete(self, session_id: str) -> None: async def delete(self, session_id: str) -> None:
await self._client.delete(self._key(session_id)) key = self._key(session_id)
payload = await self._client.get(key)
indexes: set[IndexKey] = set()
if payload is not None:
try:
indexes = set(Session.from_payload(
session_id,
self._loads(payload),
).indexes)
except Exception:
# A corrupt payload must not prevent session deletion.
pass
await self._client.delete(key)
for name, value in indexes:
await self._update_bucket(
self._index_key(name, value),
remove={session_id},
)
async def find_sessions(self, name: str, value: str) -> Sequence[str]:
index_key = self._index_key(name, value)
payload = await self._client.get(index_key)
if payload is None:
return []
members, _ = self._decode_bucket(payload)
live: list[str] = []
stale: list[str] = []
for session_id in members:
if await self._client.get(self._key(session_id)) is None:
stale.append(session_id)
else:
live.append(session_id)
if stale:
await self._update_bucket(index_key, remove=stale)
return live
@@ -0,0 +1,122 @@
import pickle
import unittest
from pwo import async_test
from test_store import FakeClock, StubMemcacheClient
from kaya.session import Session
from kaya.session.memcache import MemcacheSessionStore
class MemcacheSessionIndexTest(unittest.TestCase):
def setUp(self) -> None:
self.clock = FakeClock()
self.client = StubMemcacheClient(self.clock)
self.store = MemcacheSessionStore(self.client, clock=self.clock) # type: ignore[arg-type]
@async_test
async def test_save_load_and_find(self) -> None:
self.assertTrue(self.store.supports_indexes)
await self.store.save('abc', Session('abc', {'user': 'alice'}, indexes={('sub', 'alice')}))
loaded = await self.store.load('abc')
assert loaded is not None
self.assertEqual({'user': 'alice'}, dict(loaded))
self.assertEqual(frozenset({('sub', 'alice')}), loaded.indexes)
self.assertEqual(['abc'], await self.store.find_sessions('sub', 'alice'))
@async_test
async def test_rotation_preserves_index(self) -> None:
await self.store.save('old', Session('old', indexes={('sub', 'alice')}))
loaded = await self.store.load('old')
assert loaded is not None
loaded.set_id('new')
await self.store.delete('old')
await self.store.save('new', loaded)
self.assertEqual(['new'], await self.store.find_sessions('sub', 'alice'))
@async_test
async def test_delete_cleans_reverse_index(self) -> None:
await self.store.save('abc', Session('abc', indexes={('sub', 'alice')}))
await self.store.delete('abc')
self.assertEqual([], await self.store.find_sessions('sub', 'alice'))
@async_test
async def test_delete_by_index_deletes_all_sessions(self) -> None:
await self.store.save('a', Session('a', indexes={('sub', 'alice')}))
await self.store.save('b', Session('b', indexes={('sub', 'alice')}))
await self.store.save('c', Session('c', indexes={('sub', 'bob')}))
self.assertEqual(2, await self.store.delete_by_index('sub', 'alice'))
self.assertIsNone(await self.store.load('a'))
self.assertIsNone(await self.store.load('b'))
self.assertIsNotNone(await self.store.load('c'))
@async_test
async def test_stale_member_is_pruned(self) -> None:
await self.store.save('abc', Session('abc', indexes={('sub', 'alice')}))
await self.client.delete(b'kaya:session:abc')
self.assertEqual([], await self.store.find_sessions('sub', 'alice'))
index_key = self.store._index_key('sub', 'alice')
payload = await self.client.get(index_key)
assert payload is not None
members, _ = pickle.loads(payload)
self.assertEqual(set(), members)
@async_test
async def test_index_ttl_tracks_session_max_age(self) -> None:
await self.store.save(
'abc',
Session('abc', indexes={('sub', 'alice')}),
max_age=60,
)
index_key = self.store._index_key('sub', 'alice')
self.assertIn(index_key, self.client._expires)
self.clock.advance(61)
await self.store.save(
'abc',
Session('abc', indexes={('sub', 'alice')}),
max_age=60,
)
self.assertIsNotNone(await self.client.get(index_key))
@async_test
async def test_legacy_payload_is_upgraded_on_save(self) -> None:
await self.client.set(b'kaya:session:abc', pickle.dumps({'foo': 'bar'}))
loaded = await self.store.load('abc')
assert loaded is not None
self.assertEqual({'foo': 'bar'}, dict(loaded))
self.assertEqual(frozenset(), loaded.indexes)
loaded.add_index('sub', 'alice')
await self.store.save('abc', loaded)
raw = await self.client.get(b'kaya:session:abc')
assert raw is not None
payload = pickle.loads(raw)
self.assertEqual(1, payload['__kaya_session_v1__'])
self.assertEqual({'foo': 'bar'}, payload['data'])
self.assertEqual(['abc'], await self.store.find_sessions('sub', 'alice'))
@async_test
async def test_cas_conflict_is_retried(self) -> None:
await self.store.save('existing', Session('existing', indexes={('sub', 'alice')}))
original_cas = self.client.cas
attempts = 0
async def fail_once(key: bytes, value: bytes, cas_token: int, exptime: int = 0) -> bool:
nonlocal attempts
attempts += 1
if attempts == 1:
return False
return await original_cas(key, value, cas_token, exptime)
self.client.cas = fail_once # type: ignore[method-assign]
await self.store.save('abc', Session('abc', indexes={('sub', 'alice')}))
self.assertEqual(
{'existing', 'abc'},
set(await self.store.find_sessions('sub', 'alice')),
)
self.assertGreaterEqual(attempts, 2)
if __name__ == '__main__':
unittest.main()
@@ -28,6 +28,8 @@ class StubMemcacheClient:
self._clock = clock self._clock = clock
self._data: dict[bytes, bytes] = {} self._data: dict[bytes, bytes] = {}
self._expires: dict[bytes, float] = {} self._expires: dict[bytes, float] = {}
self._cas: dict[bytes, int] = {}
self._next_cas = 1
self.set_exptimes: list[int] = [] self.set_exptimes: list[int] = []
self.touch_exptimes: list[int] = [] self.touch_exptimes: list[int] = []
@@ -42,19 +44,46 @@ class StubMemcacheClient:
return None return None
return self._data.get(key) return self._data.get(key)
async def set(self, key: bytes, value: bytes, exptime: int = 0) -> bool: def _store(self, key: bytes, value: bytes, exptime: int) -> None:
self.set_exptimes.append(exptime)
self._data[key] = value self._data[key] = value
self._cas[key] = self._next_cas
self._next_cas += 1
if exptime > 0: if exptime > 0:
self._expires[key] = self._clock() + exptime self._expires[key] = self._clock() + exptime
else: else:
self._expires.pop(key, None) self._expires.pop(key, None)
async def set(self, key: bytes, value: bytes, exptime: int = 0) -> bool:
self.set_exptimes.append(exptime)
self._store(key, value, exptime)
return True
async def add(self, key: bytes, value: bytes, exptime: int = 0) -> bool:
if self._expired(key):
self._data.pop(key, None)
self._expires.pop(key, None)
if key in self._data:
return False
self._store(key, value, exptime)
return True
async def gets(self, key: bytes) -> tuple[Optional[bytes], Optional[int]]:
value = await self.get(key)
if value is None:
return None, None
return value, self._cas[key]
async def cas(self, key: bytes, value: bytes, cas_token: int, exptime: int = 0) -> bool:
if self._expired(key) or self._cas.get(key) != cas_token:
return False
self._store(key, value, exptime)
return True return True
async def delete(self, key: bytes) -> bool: async def delete(self, key: bytes) -> bool:
existed = key in self._data existed = key in self._data
self._data.pop(key, None) self._data.pop(key, None)
self._expires.pop(key, None) self._expires.pop(key, None)
self._cas.pop(key, None)
return existed return existed
async def touch(self, key: bytes, exptime: int) -> bool: async def touch(self, key: bytes, exptime: int) -> bool:
@@ -169,7 +198,11 @@ class MemcacheSessionStoreTest(unittest.TestCase):
clock=self.clock, clock=self.clock,
) )
await store.save('abc', Session('abc', {'foo': 'bar'})) await store.save('abc', Session('abc', {'foo': 'bar'}))
self.assertEqual(b'{"foo": "bar"}', self.client._data[b'kaya:session:abc']) self.assertEqual({
'__kaya_session_v1__': 1,
'data': {'foo': 'bar'},
'indexes': [],
}, json.loads(self.client._data[b'kaya:session:abc']))
loaded = await store.load('abc') loaded = await store.load('abc')
self.assertIsNotNone(loaded) self.assertIsNotNone(loaded)
assert loaded is not None assert loaded is not None
+12 -2
View File
@@ -30,11 +30,21 @@ Sessions are stored under keys with the prefix `kaya:session:` (configurable
via the `prefix` argument). Server-side expiry uses Redis key TTLs and slides via the `prefix` argument). Server-side expiry uses Redis key TTLs and slides
on each access when the session mixin passes a `max_age`. on each access when the session mixin passes a `max_age`.
## Session indexes
`RedisSessionStore` supports `Session.add_index()`,
`SessionStore.find_sessions()`, and `delete_by_index()`. Reverse memberships
are Redis sets updated in the same transactional pipeline as session saves.
They are removed on deletion, and stale members are pruned during lookups.
Bucket keys use a SHA-256 digest of the index name and value.
## Serialization ## Serialization
Session data is serialized with `pickle` by default, so arbitrary Python Session data is serialized with `pickle` by default, so arbitrary Python
objects can be stored. A different serializer can be plugged in via the objects can be stored. The serialized value is an envelope containing the
`dumps`/`loads` arguments: session dictionary and index metadata; legacy plain-dict values are still
accepted. A different serializer can be plugged in via the `dumps`/`loads`
arguments:
```python ```python
import json import json
@@ -26,11 +26,6 @@ dependencies = [
"redis>=5.0", "redis>=5.0",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "fakeredis"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
@@ -1,9 +1,10 @@
import hashlib
import pickle import pickle
from typing import Any, Callable, Optional, cast from typing import Any, Callable, Optional, Sequence, cast
from redis.asyncio import Redis from redis.asyncio import Redis
from kaya.session import Session, SessionStore from kaya.session import IndexKey, Session, SessionStore
class RedisSessionStore(SessionStore): class RedisSessionStore(SessionStore):
@@ -20,8 +21,14 @@ class RedisSessionStore(SessionStore):
Python objects can be stored. Custom serializers can be plugged in via Python objects can be stored. Custom serializers can be plugged in via
the ``dumps``/``loads`` arguments. Only connect this store to a trusted the ``dumps``/``loads`` arguments. Only connect this store to a trusted
Redis server, as pickle deserialization of untrusted data is unsafe. Redis server, as pickle deserialization of untrusted data is unsafe.
Reverse session indexes are stored as Redis sets. They are updated in the
same pipeline as session saves, cleaned on deletion, and pruned when stale
memberships are encountered during a lookup.
""" """
supports_indexes = True
def __init__( def __init__(
self, self,
client: Redis, client: Redis,
@@ -37,6 +44,10 @@ class RedisSessionStore(SessionStore):
def _key(self, session_id: str) -> str: def _key(self, session_id: str) -> str:
return f"{self._prefix}{session_id}" return f"{self._prefix}{session_id}"
def _index_key(self, name: str, value: str) -> str:
digest = hashlib.sha256(f"{name}\0{value}".encode()).hexdigest()
return f"{self._prefix}idx:{digest}"
async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]: async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]:
key = self._key(session_id) key = self._key(session_id)
payload = await self._client.get(key) payload = await self._client.get(key)
@@ -47,11 +58,65 @@ class RedisSessionStore(SessionStore):
await self._client.expire(key, max_age) await self._client.expire(key, max_age)
data = self._loads(cast(bytes, payload)) data = self._loads(cast(bytes, payload))
return Session(session_id, data) return Session.from_payload(session_id, data)
async def save(self, session_id: str, session: Session, max_age: Optional[int] = None) -> None: async def save(self, session_id: str, session: Session, max_age: Optional[int] = None) -> None:
payload = self._dumps(dict(session)) payload = self._dumps(session.to_payload())
await self._client.set(self._key(session_id), payload, ex=max_age) pipeline = self._client.pipeline(transaction=True)
pipeline.set(self._key(session_id), payload, ex=max_age)
for name, value in session.indexes:
index_key = self._index_key(name, value)
pipeline.sadd(index_key, session_id)
if max_age is None:
pipeline.persist(index_key)
else:
pipeline.expire(index_key, max_age + 60)
for name, value in session.removed_indexes:
pipeline.srem(self._index_key(name, value), session_id)
await pipeline.execute()
async def delete(self, session_id: str) -> None: async def delete(self, session_id: str) -> None:
await self._client.delete(self._key(session_id)) key = self._key(session_id)
payload = await self._client.get(key)
indexes: set[IndexKey] = set()
if payload is not None:
try:
indexes = set(Session.from_payload(
session_id,
self._loads(cast(bytes, payload)),
).indexes)
except Exception:
# A corrupt payload must not prevent session deletion.
pass
pipeline = self._client.pipeline(transaction=True)
pipeline.delete(key)
for name, value in indexes:
pipeline.srem(self._index_key(name, value), session_id)
await pipeline.execute()
async def find_sessions(self, name: str, value: str) -> Sequence[str]:
index_key = self._index_key(name, value)
members = await self._client.smembers(index_key)
session_ids = [
member.decode() if isinstance(member, bytes) else str(member)
for member in members
]
if not session_ids:
return []
pipeline = self._client.pipeline(transaction=False)
for session_id in session_ids:
pipeline.exists(self._key(session_id))
exists = await pipeline.execute()
live: list[str] = []
stale: list[str] = []
for session_id, found in zip(session_ids, exists):
if found:
live.append(session_id)
else:
stale.append(session_id)
if stale:
await self._client.srem(index_key, *stale)
return live
@@ -0,0 +1,94 @@
import pickle
import unittest
import fakeredis.aioredis
from pwo import async_test
from kaya.session import Session
from kaya.session.redis import RedisSessionStore
class RedisSessionIndexTest(unittest.TestCase):
def setUp(self) -> None:
self.client = fakeredis.aioredis.FakeRedis()
self.store = RedisSessionStore(self.client)
@async_test
async def test_save_load_and_find(self) -> None:
self.assertTrue(self.store.supports_indexes)
await self.store.save('abc', Session('abc', {'user': 'alice'}, indexes={('sub', 'alice')}))
loaded = await self.store.load('abc')
assert loaded is not None
self.assertEqual({'user': 'alice'}, dict(loaded))
self.assertEqual(frozenset({('sub', 'alice')}), loaded.indexes)
self.assertEqual(['abc'], await self.store.find_sessions('sub', 'alice'))
@async_test
async def test_rotation_preserves_index(self) -> None:
await self.store.save('old', Session('old', indexes={('sub', 'alice')}))
loaded = await self.store.load('old')
assert loaded is not None
loaded.set_id('new')
await self.store.delete('old')
await self.store.save('new', loaded)
self.assertEqual(['new'], await self.store.find_sessions('sub', 'alice'))
@async_test
async def test_delete_cleans_reverse_index(self) -> None:
await self.store.save('abc', Session('abc', indexes={('sub', 'alice')}))
index_key = self.store._index_key('sub', 'alice')
self.assertEqual({b'abc'}, await self.client.smembers(index_key))
await self.store.delete('abc')
self.assertEqual(set(), await self.client.smembers(index_key))
@async_test
async def test_delete_by_index_deletes_all_sessions(self) -> None:
await self.store.save('a', Session('a', indexes={('sub', 'alice')}))
await self.store.save('b', Session('b', indexes={('sub', 'alice')}))
await self.store.save('c', Session('c', indexes={('sub', 'bob')}))
self.assertEqual(2, await self.store.delete_by_index('sub', 'alice'))
self.assertIsNone(await self.store.load('a'))
self.assertIsNone(await self.store.load('b'))
self.assertIsNotNone(await self.store.load('c'))
@async_test
async def test_stale_member_is_pruned(self) -> None:
await self.store.save('abc', Session('abc', indexes={('sub', 'alice')}))
await self.client.delete('kaya:session:abc')
self.assertEqual([], await self.store.find_sessions('sub', 'alice'))
index_key = self.store._index_key('sub', 'alice')
self.assertEqual(set(), await self.client.smembers(index_key))
@async_test
async def test_index_ttl_tracks_session_max_age(self) -> None:
await self.store.save(
'abc',
Session('abc', indexes={('sub', 'alice')}),
max_age=60,
)
index_key = self.store._index_key('sub', 'alice')
ttl = await self.client.ttl(index_key)
self.assertGreater(ttl, 60)
self.assertLessEqual(ttl, 120)
@async_test
async def test_legacy_payload_is_upgraded_on_save(self) -> None:
await self.client.set('kaya:session:abc', pickle.dumps({'foo': 'bar'}))
loaded = await self.store.load('abc')
assert loaded is not None
self.assertEqual({'foo': 'bar'}, dict(loaded))
self.assertEqual(frozenset(), loaded.indexes)
loaded.add_index('sub', 'alice')
await self.store.save('abc', loaded)
raw = await self.client.get('kaya:session:abc')
assert raw is not None
payload = pickle.loads(raw)
self.assertEqual(1, payload['__kaya_session_v1__'])
self.assertEqual({'foo': 'bar'}, payload['data'])
self.assertEqual(['abc'], await self.store.find_sessions('sub', 'alice'))
if __name__ == '__main__':
unittest.main()
@@ -89,7 +89,13 @@ class RedisSessionStoreTest(unittest.TestCase):
loads=lambda b: json.loads(b.decode('utf-8')), loads=lambda b: json.loads(b.decode('utf-8')),
) )
await store.save('abc', Session('abc', {'foo': 'bar'})) await store.save('abc', Session('abc', {'foo': 'bar'}))
self.assertEqual(b'{"foo": "bar"}', await self.client.get('kaya:session:abc')) raw = await self.client.get('kaya:session:abc')
assert raw is not None
self.assertEqual({
'__kaya_session_v1__': 1,
'data': {'foo': 'bar'},
'indexes': [],
}, json.loads(raw))
loaded = await store.load('abc') loaded = await store.load('abc')
self.assertIsNotNone(loaded) self.assertIsNotNone(loaded)
assert loaded is not None assert loaded is not None
+32
View File
@@ -66,6 +66,37 @@ session and creates a fresh empty one.
Set `max_age=None` to disable server-side expiry (and the `Max-Age` cookie Set `max_age=None` to disable server-side expiry (and the `Max-Age` cookie
attribute) entirely. attribute) entirely.
## Session indexes
Authentication layers can attach reverse-index metadata to a session without
adding it to the user-visible session dictionary:
```python
ctx.session.add_index('oidc:default:sub', 'user-123')
```
Index-capable stores persist these memberships on `save()` and maintain them
across session ID rotation and deletion:
```python
session_ids = await store.find_sessions('oidc:default:sub', 'user-123')
deleted = await store.delete_by_index('oidc:default:sub', 'user-123')
```
Use `session.remove_index(name, value)` to remove a membership while keeping
the session. `SessionMixin.store` exposes the backing store for features such
as back-channel logout.
`InMemorySessionStore`, `RedisSessionStore`, and `MemcacheSessionStore` set
`supports_indexes = True`. Custom stores can opt in by setting that flag,
implementing `find_sessions()`, maintaining reverse memberships on `save()`,
and removing them on `delete()`. `delete_by_index()` has a generic default
implementation based on `find_sessions()` and `delete()`.
Remote stores serialize an envelope containing both the session dictionary and
its index metadata. Existing plain-dict payloads remain readable and are
upgraded on the next save.
## Features ## Features
- `Session`: dict-like session object with modification tracking - `Session`: dict-like session object with modification tracking
@@ -74,6 +105,7 @@ attribute) entirely.
- `SessionMixin`: composable Kaya mixin managing session cookies and persistence - `SessionMixin`: composable Kaya mixin managing session cookies and persistence
- Session ID regeneration (`session.regenerate_id()`) and invalidation - Session ID regeneration (`session.regenerate_id()`) and invalidation
(`session.invalidate()`) for authentication layers (`session.invalidate()`) for authentication layers
- Reverse session indexes for looking up and invalidating sessions by metadata
- WebSocket support: the session is exposed as `ws.session` in websocket - WebSocket support: the session is exposed as `ws.session` in websocket
handlers, loaded at connect time and persisted on close handlers, loaded at connect time and persisted on close
-5
View File
@@ -26,11 +26,6 @@ dependencies = [
"pwo", "pwo",
] ]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws"
]
[project.urls] [project.urls]
"Homepage" = "https://github.com/woggioni/kaya" "Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues" "Bug Tracker" = "https://github.com/woggioni/kaya/issues"
@@ -3,12 +3,13 @@ from pkgutil import extend_path
__path__ = extend_path(__path__, __name__) __path__ = extend_path(__path__, __name__)
from ._mixin import SessionMixin from ._mixin import SessionMixin
from ._session import Session from ._session import IndexKey, Session
from ._store import InMemorySessionStore, SessionStore from ._store import InMemorySessionStore, SessionStore
__all__ = [ __all__ = [
'InMemorySessionStore', 'InMemorySessionStore',
'IndexKey',
'Session', 'Session',
'SessionMixin', 'SessionMixin',
'SessionStore', 'SessionStore',
@@ -183,6 +183,16 @@ class SessionMixin(KayaMixin):
self._secure = secure self._secure = secure
self._samesite = samesite self._samesite = samesite
@property
def store(self) -> SessionStore:
"""Return the backing session store."""
return self._store
@property
def max_age(self) -> Optional[int]:
"""Return the idle timeout used when persisting sessions."""
return self._max_age
def apply(self, app: KayaApp) -> None: def apply(self, app: KayaApp) -> None:
app.add_before_request_hook(self._before_request) app.add_before_request_hook(self._before_request)
app.add_after_request_hook(self._after_request) app.add_after_request_hook(self._after_request)
@@ -273,3 +283,4 @@ class SessionMixin(KayaMixin):
if session.id is not None: if session.id is not None:
await self._store.save(session.id, session, self._max_age) await self._store.save(session.id, session, self._max_age)
session.commit_indexes()
@@ -1,4 +1,10 @@
from typing import Any, Iterator, Mapping, MutableMapping, Optional from typing import Any, Iterable, Iterator, Mapping, MutableMapping, Optional
IndexKey = tuple[str, str]
_PAYLOAD_MARKER = '__kaya_session_v1__'
_PAYLOAD_VERSION = 1
class Session(MutableMapping[str, Any]): class Session(MutableMapping[str, Any]):
@@ -9,13 +15,21 @@ class Session(MutableMapping[str, Any]):
it. it.
""" """
def __init__(self, session_id: Optional[str] = None, data: Optional[Mapping[str, Any]] = None) -> None: def __init__(
self,
session_id: Optional[str] = None,
data: Optional[Mapping[str, Any]] = None,
indexes: Iterable[IndexKey] = (),
) -> None:
self._id: Optional[str] = session_id self._id: Optional[str] = session_id
self._old_id: Optional[str] = None self._old_id: Optional[str] = None
self._data: dict[str, Any] = dict(data) if data else {} self._data: dict[str, Any] = dict(data) if data else {}
self._modified: bool = False self._modified: bool = False
self._regenerate: bool = False self._regenerate: bool = False
self._invalidated: bool = False self._invalidated: bool = False
self._persisted_indexes: set[IndexKey] = set(indexes)
self._added_indexes: set[IndexKey] = set()
self._removed_indexes: set[IndexKey] = set()
@property @property
def id(self) -> Optional[str]: def id(self) -> Optional[str]:
@@ -60,6 +74,75 @@ class Session(MutableMapping[str, Any]):
def mark_modified(self) -> None: def mark_modified(self) -> None:
self._modified = True self._modified = True
def add_index(self, name: str, value: str) -> None:
"""Index this session under ``(name, value)`` when it is saved.
Index-capable stores can use this metadata to find and invalidate
sessions without knowing their IDs. Adding an already persisted index
is harmless and does not create a duplicate membership.
"""
index = (name, value)
self._removed_indexes.discard(index)
if index not in self._persisted_indexes:
self._added_indexes.add(index)
self._modified = True
def remove_index(self, name: str, value: str) -> None:
"""Remove an index membership when this session is saved."""
index = (name, value)
if index in self._added_indexes:
self._added_indexes.discard(index)
elif index in self._persisted_indexes:
self._removed_indexes.add(index)
self._modified = True
@property
def indexes(self) -> frozenset[IndexKey]:
"""Return the index memberships that should be persisted."""
return frozenset(
(self._persisted_indexes - self._removed_indexes) | self._added_indexes
)
@property
def added_indexes(self) -> frozenset[IndexKey]:
"""Return index memberships added since the last successful save."""
return frozenset(self._added_indexes)
@property
def removed_indexes(self) -> frozenset[IndexKey]:
"""Return index memberships removed since the last successful save."""
return frozenset(self._removed_indexes)
def commit_indexes(self) -> None:
"""Mark the current index memberships as persisted."""
self._persisted_indexes = set(self.indexes)
self._added_indexes.clear()
self._removed_indexes.clear()
def to_payload(self) -> dict[str, Any]:
"""Return the serializable session payload used by remote stores."""
return {
_PAYLOAD_MARKER: _PAYLOAD_VERSION,
'data': dict(self),
'indexes': sorted(self.indexes),
}
@classmethod
def from_payload(cls, session_id: str, payload: Mapping[str, Any]) -> 'Session':
"""Build a session from an envelope or a legacy plain-data payload."""
data = payload.get('data')
raw_indexes = payload.get('indexes')
if (payload.get(_PAYLOAD_MARKER) == _PAYLOAD_VERSION
and isinstance(data, Mapping)
and isinstance(raw_indexes, (list, tuple))):
indexes: set[IndexKey] = set()
for item in raw_indexes:
if (isinstance(item, (list, tuple)) and len(item) == 2
and isinstance(item[0], str) and isinstance(item[1], str)):
indexes.add((item[0], item[1]))
return cls(session_id, data, indexes=indexes)
return cls(session_id, payload)
def __getitem__(self, key: str) -> Any: def __getitem__(self, key: str) -> Any:
return self._data[key] return self._data[key]
@@ -1,13 +1,21 @@
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from secrets import token_urlsafe from secrets import token_urlsafe
from time import monotonic from time import monotonic
from typing import Any, Callable, Optional from typing import Any, Callable, ClassVar, Optional, Sequence
from ._session import Session from ._session import IndexKey, Session
class SessionStore(ABC): class SessionStore(ABC):
"""Pluggable backend for session persistence.""" """Pluggable backend for session persistence.
Stores with ``supports_indexes = True`` also maintain reverse indexes for
:class:`~kaya.session.Session` index metadata. They must add the session ID
to every effective index on ``save()``, apply pending removals, and remove
all of a session's index memberships on ``delete()``.
"""
supports_indexes: ClassVar[bool] = False
@abstractmethod @abstractmethod
async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]: async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]:
@@ -33,6 +41,23 @@ class SessionStore(ABC):
"""Remove the session from the store.""" """Remove the session from the store."""
pass pass
async def find_sessions(self, name: str, value: str) -> Sequence[str]:
"""Return live session IDs indexed under ``(name, value)``.
Only available on stores whose ``supports_indexes`` flag is true.
Implementations should prune stale index memberships encountered here.
"""
raise NotImplementedError(
f'{type(self).__name__} does not support session indexes'
)
async def delete_by_index(self, name: str, value: str) -> int:
"""Delete every live session indexed under ``(name, value)``."""
session_ids = await self.find_sessions(name, value)
for session_id in session_ids:
await self.delete(session_id)
return len(session_ids)
def new_session_id(self) -> str: def new_session_id(self) -> str:
"""Return a new opaque session identifier. """Return a new opaque session identifier.
@@ -51,10 +76,26 @@ class InMemorySessionStore(SessionStore):
inactivity. Active sessions slide the expiry window on each access. inactivity. Active sessions slide the expiry window on each access.
""" """
supports_indexes = True
def __init__(self, clock: Callable[[], float] = monotonic) -> None: def __init__(self, clock: Callable[[], float] = monotonic) -> None:
self._clock = clock self._clock = clock
self._data: dict[str, dict[str, Any]] = {} self._data: dict[str, dict[str, Any]] = {}
self._expires: dict[str, float] = {} self._expires: dict[str, float] = {}
self._session_indexes: dict[str, set[IndexKey]] = {}
self._index_sessions: dict[IndexKey, set[str]] = {}
def _drop_session(self, session_id: str) -> None:
self._data.pop(session_id, None)
self._expires.pop(session_id, None)
indexes = self._session_indexes.pop(session_id, set())
for index in indexes:
session_ids = self._index_sessions.get(index)
if session_ids is None:
continue
session_ids.discard(session_id)
if not session_ids:
self._index_sessions.pop(index, None)
async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]: async def load(self, session_id: str, max_age: Optional[int] = None) -> Optional[Session]:
data = self._data.get(session_id) data = self._data.get(session_id)
@@ -64,14 +105,17 @@ class InMemorySessionStore(SessionStore):
expires = self._expires.get(session_id) expires = self._expires.get(session_id)
now = self._clock() now = self._clock()
if expires is not None and now > expires: if expires is not None and now > expires:
self._data.pop(session_id, None) self._drop_session(session_id)
self._expires.pop(session_id, None)
return None return None
if max_age is not None and expires is not None: if max_age is not None and expires is not None:
self._expires[session_id] = now + max_age self._expires[session_id] = now + max_age
return Session(session_id, data) return Session(
session_id,
data,
indexes=self._session_indexes.get(session_id, ()),
)
async def save(self, session_id: str, session: Session, max_age: Optional[int] = None) -> None: async def save(self, session_id: str, session: Session, max_age: Optional[int] = None) -> None:
self._data[session_id] = dict(session) self._data[session_id] = dict(session)
@@ -80,6 +124,34 @@ class InMemorySessionStore(SessionStore):
else: else:
self._expires.pop(session_id, None) self._expires.pop(session_id, None)
for index in session.removed_indexes:
session_ids = self._index_sessions.get(index)
if session_ids is None:
continue
session_ids.discard(session_id)
if not session_ids:
self._index_sessions.pop(index, None)
indexes = set(session.indexes)
self._session_indexes[session_id] = indexes
for index in indexes:
self._index_sessions.setdefault(index, set()).add(session_id)
async def delete(self, session_id: str) -> None: async def delete(self, session_id: str) -> None:
self._data.pop(session_id, None) self._drop_session(session_id)
self._expires.pop(session_id, None)
async def find_sessions(self, name: str, value: str) -> Sequence[str]:
session_ids = self._index_sessions.get((name, value))
if session_ids is None:
return []
now = self._clock()
live: list[str] = []
for session_id in list(session_ids):
expires = self._expires.get(session_id)
if (session_id not in self._data
or (expires is not None and now > expires)):
self._drop_session(session_id)
else:
live.append(session_id)
return live
+169
View File
@@ -0,0 +1,169 @@
import unittest
import httpx
from pwo import async_test
from kaya.core import HttpContext, KayaApp
from kaya.session import InMemorySessionStore, Session, SessionMixin
class FakeClock:
def __init__(self, start: float = 0.0) -> None:
self._now = start
def __call__(self) -> float:
return self._now
def advance(self, seconds: float) -> None:
self._now += seconds
class SessionIndexUnitTest(unittest.TestCase):
def test_add_and_remove_index(self) -> None:
session = Session()
session.add_index('sub', 'alice')
self.assertEqual(frozenset({('sub', 'alice')}), session.indexes)
self.assertEqual(frozenset({('sub', 'alice')}), session.added_indexes)
self.assertTrue(session.modified)
session.remove_index('sub', 'alice')
self.assertEqual(frozenset(), session.indexes)
self.assertEqual(frozenset(), session.added_indexes)
self.assertEqual(frozenset(), session.removed_indexes)
def test_remove_persisted_index(self) -> None:
session = Session(indexes={('sub', 'alice')})
self.assertFalse(session.modified)
session.remove_index('sub', 'alice')
self.assertEqual(frozenset(), session.indexes)
self.assertEqual(frozenset({('sub', 'alice')}), session.removed_indexes)
session.add_index('sub', 'alice')
self.assertEqual(frozenset({('sub', 'alice')}), session.indexes)
self.assertEqual(frozenset(), session.removed_indexes)
def test_payload_round_trip(self) -> None:
session = Session('abc', {'user': 'alice'}, indexes={('sub', 'alice')})
loaded = Session.from_payload('abc', session.to_payload())
self.assertEqual({'user': 'alice'}, dict(loaded))
self.assertEqual(frozenset({('sub', 'alice')}), loaded.indexes)
def test_legacy_payload(self) -> None:
loaded = Session.from_payload('abc', {'user': 'alice'})
self.assertEqual({'user': 'alice'}, dict(loaded))
self.assertEqual(frozenset(), loaded.indexes)
class InMemorySessionIndexTest(unittest.TestCase):
@async_test
async def test_save_load_and_find(self) -> None:
store = InMemorySessionStore()
self.assertTrue(store.supports_indexes)
session = Session('abc', {'user': 'alice'})
session.add_index('sub', 'alice')
await store.save('abc', session)
session.commit_indexes()
loaded = await store.load('abc')
assert loaded is not None
self.assertEqual(frozenset({('sub', 'alice')}), loaded.indexes)
self.assertEqual(['abc'], await store.find_sessions('sub', 'alice'))
self.assertEqual([], await store.find_sessions('sub', 'bob'))
@async_test
async def test_delete_cleans_reverse_index(self) -> None:
store = InMemorySessionStore()
session = Session('abc', indexes={('sub', 'alice')})
await store.save('abc', session)
await store.delete('abc')
self.assertEqual([], await store.find_sessions('sub', 'alice'))
self.assertEqual({}, store._session_indexes)
@async_test
async def test_delete_by_index_deletes_all_sessions(self) -> None:
store = InMemorySessionStore()
await store.save('a', Session('a', indexes={('sub', 'alice')}))
await store.save('b', Session('b', indexes={('sub', 'alice')}))
await store.save('c', Session('c', indexes={('sub', 'bob')}))
self.assertEqual(2, await store.delete_by_index('sub', 'alice'))
self.assertIsNone(await store.load('a'))
self.assertIsNone(await store.load('b'))
self.assertIsNotNone(await store.load('c'))
@async_test
async def test_expiry_prunes_index(self) -> None:
clock = FakeClock()
store = InMemorySessionStore(clock=clock)
await store.save(
'abc',
Session('abc', indexes={('sub', 'alice')}),
max_age=60,
)
clock.advance(61)
self.assertEqual([], await store.find_sessions('sub', 'alice'))
self.assertEqual({}, store._session_indexes)
@async_test
async def test_removed_index_is_not_readded(self) -> None:
store = InMemorySessionStore()
session = Session('abc', indexes={('sub', 'alice')})
await store.save('abc', session)
session.commit_indexes()
session.remove_index('sub', 'alice')
await store.save('abc', session)
self.assertEqual([], await store.find_sessions('sub', 'alice'))
self.assertEqual(frozenset(), session.indexes)
class SessionIndexIntegrationTest(unittest.TestCase):
def setUp(self) -> None:
self.store = InMemorySessionStore()
self.mixin = SessionMixin(self.store)
self.app = KayaApp(mixins=[self.mixin])
@self.app.GET('/login')
async def login(ctx: HttpContext) -> None:
ctx.session['user'] = 'alice'
ctx.session.add_index('sub', 'alice')
await ctx.send_str(200, 'ok')
@self.app.GET('/rotate')
async def rotate(ctx: HttpContext) -> None:
ctx.session.regenerate_id()
await ctx.send_str(200, 'ok')
@self.app.GET('/logout')
async def logout(ctx: HttpContext) -> None:
ctx.session.invalidate()
await ctx.send_str(200, 'ok')
@async_test
async def test_rotation_preserves_index(self) -> None:
self.assertIs(self.store, self.mixin.store)
transport = httpx.ASGITransport(app=self.app)
async with httpx.AsyncClient(transport=transport, base_url='http://127.0.0.1:80') as client:
await client.get('/login')
old_id = client.cookies['session_id']
self.assertEqual([old_id], await self.store.find_sessions('sub', 'alice'))
await client.get('/rotate')
new_id = client.cookies['session_id']
self.assertNotEqual(old_id, new_id)
self.assertEqual([new_id], await self.store.find_sessions('sub', 'alice'))
@async_test
async def test_invalidation_cleans_index(self) -> None:
transport = httpx.ASGITransport(app=self.app)
async with httpx.AsyncClient(transport=transport, base_url='http://127.0.0.1:80') as client:
await client.get('/login')
await client.get('/logout')
self.assertEqual([], await self.store.find_sessions('sub', 'alice'))
if __name__ == '__main__':
unittest.main()
+51
View File
@@ -0,0 +1,51 @@
[project]
name = "kaya"
version = "0.0.0"
description = "Development workspace for the Kaya lightweight ASGI web framework"
requires-python = ">=3.10"
dependencies = [
"kaya-core",
"kaya-rsgi",
"kaya-session",
"kaya-session-redis",
"kaya-session-memcache",
"kaya-oidc",
"kaya-openapi",
"kaya-cors",
"kaya-forwarded",
"kaya-otel",
]
[dependency-groups]
dev = [
"fakeredis",
"mypy",
"ipdb",
"httpx",
"httpx-ws",
]
[tool.uv]
package = false
required-version = ">=0.13"
[tool.uv.workspace]
members = ["packages/*"]
[tool.uv.sources]
kaya-core = { workspace = true }
kaya-rsgi = { workspace = true }
kaya-session = { workspace = true }
kaya-session-redis = { workspace = true }
kaya-session-memcache = { workspace = true }
kaya-oidc = { workspace = true }
kaya-openapi = { workspace = true }
kaya-cors = { workspace = true }
kaya-forwarded = { workspace = true }
kaya-otel = { workspace = true }
pwo = { index = "gitea" }
[[tool.uv.index]]
name = "gitea"
url = "https://gitea.woggioni.net/api/packages/woggioni/pypi/simple"
explicit = true
-16
View File
@@ -1,16 +0,0 @@
kaya-core @ file:./packages/kaya-core
kaya-rsgi @ file:./packages/kaya-rsgi
kaya-session @ file:./packages/kaya-session
kaya-session-redis @ file:./packages/kaya-session-redis
kaya-session-memcache @ file:./packages/kaya-session-memcache
kaya-oidc @ file:./packages/kaya-oidc
kaya-openapi @ file:./packages/kaya-openapi
kaya-cors @ file:./packages/kaya-cors
kaya-forwarded @ file:./packages/kaya-forwarded
build
fakeredis
mypy
ipdb
twine
httpx
httpx-ws
-214
View File
@@ -1,214 +0,0 @@
#
# This file is autogenerated by pip-compile with Python 3.14
# by the following command:
#
# pip-compile --allow-unsafe --extra-index-url=https://pypi.org/simple --index-url=https://gitea.woggioni.net/api/packages/woggioni/pypi/simple --output-file=requirements-dev.txt requirements-dev.in
#
--index-url https://gitea.woggioni.net/api/packages/woggioni/pypi/simple
--extra-index-url https://pypi.org/simple
aiomcache==0.8.2
# via kaya-session-memcache
anyio==4.14.2
# via
# httpx
# httpx-ws
ast-serialize==0.6.0
# via mypy
asttokens==3.0.2
# via stack-data
build==1.5.0
# via -r requirements-dev.in
certifi==2026.6.17
# via
# httpcore
# httpx
# requests
cffi==2.1.0
# via cryptography
charset-normalizer==3.4.9
# via requests
click==8.4.2
# via granian
cryptography==49.0.0
# via
# pyjwt
# secretstorage
decorator==5.3.1
# via
# ipdb
# ipython
docutils==0.23
# via readme-renderer
executing==2.2.1
# via stack-data
fakeredis==2.36.2
# via -r requirements-dev.in
granian==2.7.9
# via kaya-rsgi
h11==0.16.0
# via
# httpcore
# wsproto
httpcore==1.0.9
# via
# httpx
# httpx-ws
httpx==0.28.1
# via
# -r requirements-dev.in
# httpx-ws
# kaya-oidc
httpx-ws==0.9.0
# via -r requirements-dev.in
id==1.6.1
# via twine
idna==3.18
# via
# anyio
# httpx
# requests
ipdb==0.13.13
# via -r requirements-dev.in
ipython==9.15.0
# via ipdb
ipython-pygments-lexers==1.1.1
# via ipython
jaraco-classes==3.4.0
# via keyring
jaraco-context==6.1.2
# via keyring
jaraco-functools==4.6.0
# via keyring
jedi==0.20.0
# via ipython
jeepney==0.9.0
# via
# keyring
# secretstorage
file:./packages/kaya-core
# via
# -r requirements-dev.in
# kaya-cors
# kaya-forwarded
# kaya-oidc
# kaya-openapi
# kaya-rsgi
# kaya-session
file:./packages/kaya-cors
# via -r requirements-dev.in
file:./packages/kaya-forwarded
# via -r requirements-dev.in
file:./packages/kaya-oidc
# via -r requirements-dev.in
file:./packages/kaya-openapi
# via -r requirements-dev.in
file:./packages/kaya-rsgi
# via -r requirements-dev.in
file:./packages/kaya-session
# via
# -r requirements-dev.in
# kaya-oidc
# kaya-session-memcache
# kaya-session-redis
file:./packages/kaya-session-memcache
# via -r requirements-dev.in
file:./packages/kaya-session-redis
# via -r requirements-dev.in
keyring==25.7.0
# via twine
librt==0.13.0
# via mypy
markdown-it-py==4.2.0
# via rich
matplotlib-inline==0.2.2
# via ipython
mdurl==0.1.2
# via markdown-it-py
more-itertools==11.1.0
# via
# jaraco-classes
# jaraco-functools
mypy==2.3.0
# via -r requirements-dev.in
mypy-extensions==1.1.0
# via mypy
nh3==0.3.6
# via readme-renderer
packaging==26.2
# via
# build
# twine
parso==0.8.7
# via jedi
pathspec==1.1.1
# via mypy
pexpect==4.9.0
# via ipython
prompt-toolkit==3.0.52
# via ipython
psutil==7.2.2
# via ipython
ptyprocess==0.7.0
# via pexpect
pure-eval==0.2.3
# via stack-data
pwo==0.1.2
# via
# kaya-core
# kaya-rsgi
# kaya-session
pycparser==3.0
# via cffi
pygments==2.20.0
# via
# ipython
# ipython-pygments-lexers
# readme-renderer
# rich
pyjwt[crypto]==2.13.0
# via kaya-oidc
pyproject-hooks==1.2.0
# via build
readme-renderer==45.0
# via twine
redis==8.0.1
# via
# fakeredis
# kaya-session-redis
requests==2.34.2
# via
# requests-toolbelt
# twine
requests-toolbelt==1.0.0
# via twine
rfc3986==2.0.0
# via twine
rich==15.0.0
# via twine
secretstorage==3.5.0
# via keyring
sortedcontainers==2.4.0
# via fakeredis
stack-data==0.6.3
# via ipython
traitlets==5.15.1
# via
# ipython
# matplotlib-inline
twine==6.2.0
# via -r requirements-dev.in
typing-extensions==4.16.0
# via
# kaya-core
# mypy
# pwo
urllib3==2.7.0
# via
# id
# requests
# twine
wcwidth==0.8.2
# via prompt-toolkit
wsproto==1.3.2
# via httpx-ws
Generated
+1669
View File
File diff suppressed because it is too large. Load diff