Author SHA1 Message Date
woggioni 148a35b71c Add kaya-otel package for OpenTelemetry tracing and metrics
CI / Build Pip package (push) Successful in 4m8s
Instrument HTTP requests and WebSocket connections via Kaya hooks, covering both ASGI and RSGI. Records handler exceptions, WebSocket close codes, optional header capture, exclusions and lifecycle hooks.

Add route-template resolution and exception visibility to kaya-core so trace/metric attributes can use low-cardinality routes and failed spans can record escaped exceptions.
2026-09-19 00:39:15 +00:00
woggioni 69762d93df Document that mixin setup/shutdown loops may not be running
Under RSGI __rsgi_init__ is called before the loop starts, so
asyncio.get_running_loop() raises RuntimeError inside setup(). Spell out
that mixins must use the loop they are given, since the ASGI lifespan
incidentally runs with a live loop.
2026-09-18 02:37:08 +00:00
woggioni aa35e2d30c Refactor forwarded header handling into opt-in kaya-forwarded package with trusted CIDRs
CI / Build Pip package (push) Successful in 3m59s
2026-09-05 15:11:05 +08:00
woggioni 850d5dda35 Add kaya-cors check and publish steps to CI pipeline
CI / Build Pip package (push) Successful in 3m6s
2026-09-04 16:29:52 +08:00
woggioni 270a0d87fc Add kaya-cors package for CORS support
CI / Build Pip package (push) Successful in 3m44s
2026-09-04 15:25:26 +08:00
woggioni 59f1a8227f Resolve client address from Forwarded and X-Forwarded-* headers 2026-09-04 15:25:18 +08:00
woggioni 9a68d10868 Add kaya-openapi package for automatic OpenAPI spec generation
- New packages/kaya-openapi with OpenAPIMixin, @operation decorator,
  and generate_spec() that walks the routing tree
- Enables kaya-core's Tree.register to expose the original handler
  callback as an instance attribute for metadata introspection
- Registers GET /openapi.json and GET /docs (Swagger UI) routes
- Supports  and  path parameters, docstring
  descriptions, @operation metadata, and excludes wildcard/WS routes
- Adds example/openapi.py, updates CI, README, and requirements
2026-07-25 09:20:19 +00:00
woggioni 87d1b0acb1 Fix key-loop in Tree.add to reuse existing children for literal segments after a matcher boundary
The key-loop unconditionally called self.parse() and overwrote
result.children[key] for literal segments after the walk-down loop
broke at a parameter. This destroyed pre-existing subtrees (with their
method children and handlers) when a second route (e.g. POST) shared
the same literal sub-segments after a parameter.

Added a children.get(key) reuse check before parse(), mirroring the
walk-down loop's child-reuse logic but extended past the boundary
where parameters live in path_matchers, not children.
2026-07-24 12:53:58 +00:00
woggioni dfc5d70eec Allow nested routes sharing the same path parameter matcher
When two routes share the same parameter at the same node position
(e.g. GET /restaurants/${id} and GET /restaurants/${id}/menu),
reuse the existing equivalent matcher instead of raising a conflict.
Non-equivalent matchers (different names, kinds, or glob patterns)
at the same node for the same method still raise ValueError.
2026-07-24 06:46:07 +00:00
woggioni 08370c4963 Rename kaya.session_redis → kaya.session.redis
- Move src/kaya/session_redis/ → src/kaya/session/redis/
- Update pyproject.toml version_file path
- Update all import references (tests, READMEs, root README)
2026-07-23 16:02:56 +00:00
woggioni 01a54e3fb5 Rename kaya.session_memcache → kaya.session.memcache
- Move src/kaya/session_memcache/ → src/kaya/session/memcache/
- Add pkgutil.extend_path to kaya.session for subpackage namespace support
- Update pyproject.toml version_file path
- Update all import references (tests, READMEs, root README)
2026-07-23 15:51:42 +00:00
51 changed files with 3461 additions and 13 deletions
+48
View File
@@ -48,6 +48,22 @@ jobs:
run: |
.venv/bin/python -m mypy -p kaya.oidc
.venv/bin/python -m unittest discover -s packages/kaya-oidc/tests
- name: Check kaya-openapi
run: |
.venv/bin/python -m mypy -p kaya.openapi
.venv/bin/python -m unittest discover -s packages/kaya-openapi/tests
- name: Check kaya-cors
run: |
.venv/bin/python -m mypy -p kaya.cors
.venv/bin/python -m unittest discover -s packages/kaya-cors/tests
- name: Check kaya-forwarded
run: |
.venv/bin/python -m mypy -p kaya.forwarded
.venv/bin/python -m unittest discover -s packages/kaya-forwarded/tests
- name: Check kaya-otel
run: |
.venv/bin/python -m mypy -p kaya.otel
.venv/bin/python -m unittest discover -s packages/kaya-otel/tests
- name: Publish kaya-core artifacts
env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }}
@@ -96,3 +112,35 @@ jobs:
run: |
.venv/bin/pyproject-build packages/kaya-oidc
.venv/bin/twine upload --repository gitea packages/kaya-oidc/dist/*.whl packages/kaya-oidc/dist/*.tar.gz
- name: Publish kaya-openapi artifacts
env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: |
.venv/bin/pyproject-build packages/kaya-openapi
.venv/bin/twine upload --repository gitea packages/kaya-openapi/dist/*.whl packages/kaya-openapi/dist/*.tar.gz
- name: Publish kaya-cors artifacts
env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: |
.venv/bin/pyproject-build packages/kaya-cors
.venv/bin/twine upload --repository gitea packages/kaya-cors/dist/*.whl packages/kaya-cors/dist/*.tar.gz
- name: Publish kaya-forwarded artifacts
env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: |
.venv/bin/pyproject-build packages/kaya-forwarded
.venv/bin/twine upload --repository gitea packages/kaya-forwarded/dist/*.whl packages/kaya-forwarded/dist/*.tar.gz
- name: Publish kaya-otel artifacts
env:
TWINE_REPOSITORY_URL: ${{ vars.PYPI_REGISTRY_URL }}
TWINE_USERNAME: ${{ vars.PUBLISHER_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PUBLISHER_TOKEN }}
run: |
.venv/bin/pyproject-build packages/kaya-otel
.venv/bin/twine upload --repository gitea packages/kaya-otel/dist/*.whl packages/kaya-otel/dist/*.tar.gz
+19 -3
View File
@@ -12,6 +12,10 @@ This repository is a monorepo for the Kaya framework. The code is split into ind
- **kaya-session-redis** — Redis-backed session storage (`packages/kaya-session-redis/`)
- **kaya-session-memcache** — memcached-backed session storage (`packages/kaya-session-memcache/`)
- **kaya-oidc** — OpenID Connect authentication (`packages/kaya-oidc/`)
- **kaya-openapi** — automatic OpenAPI specification generation (`packages/kaya-openapi/`)
- **kaya-cors** — CORS (Cross-Origin Resource Sharing) support (`packages/kaya-cors/`)
- **kaya-forwarded** — trusted-proxy `Forwarded`/`X-Forwarded-*` client address resolution (`packages/kaya-forwarded/`)
- **kaya-otel** — OpenTelemetry tracing and metrics (`packages/kaya-otel/`)
Additional `kaya-*` packages can be added as new directories under `packages/`.
@@ -26,7 +30,7 @@ pip install --index-url https://gitea.woggioni.net/api/packages/woggioni/pypi/si
Install the packages in development mode:
```bash
pip install -e packages/kaya-core -e packages/kaya-rsgi -e packages/kaya-session -e packages/kaya-session-redis -e packages/kaya-session-memcache -e packages/kaya-oidc
pip install -e packages/kaya-core -e packages/kaya-rsgi -e packages/kaya-session -e packages/kaya-session-redis -e packages/kaya-session-memcache -e packages/kaya-oidc -e packages/kaya-openapi -e packages/kaya-cors -e packages/kaya-forwarded -e packages/kaya-otel
```
Run the example:
@@ -44,6 +48,10 @@ python -m unittest discover -s packages/kaya-session/tests
python -m unittest discover -s packages/kaya-session-redis/tests
python -m unittest discover -s packages/kaya-session-memcache/tests
python -m unittest discover -s packages/kaya-oidc/tests
python -m unittest discover -s packages/kaya-openapi/tests
python -m unittest discover -s packages/kaya-cors/tests
python -m unittest discover -s packages/kaya-forwarded/tests
python -m unittest discover -s packages/kaya-otel/tests
```
## Static analysis
@@ -52,9 +60,13 @@ python -m unittest discover -s packages/kaya-oidc/tests
mypy -p kaya.core
mypy -p kaya.rsgi
mypy -p kaya.session
mypy -p kaya.session_redis
mypy -p kaya.session_memcache
mypy -p kaya.session.redis
mypy -p kaya.session.memcache
mypy -p kaya.oidc
mypy -p kaya.openapi
mypy -p kaya.cors
mypy -p kaya.forwarded
mypy -p kaya.otel
```
## Building packages
@@ -66,4 +78,8 @@ python -m build packages/kaya-session
python -m build packages/kaya-session-redis
python -m build packages/kaya-session-memcache
python -m build packages/kaya-oidc
python -m build packages/kaya-openapi
python -m build packages/kaya-cors
python -m build packages/kaya-forwarded
python -m build packages/kaya-otel
```
+32
View File
@@ -0,0 +1,32 @@
from kaya.core import HttpContext, KayaApp
from kaya.openapi import OpenAPIMixin, operation
app = KayaApp(mixins=[OpenAPIMixin(
title='Greeting API',
version='1.0.0',
description='Example API documented with kaya-openapi',
)])
@app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
"""Say hello to the world."""
await ctx.send_str(200, 'Hello World')
@app.GET('/hello/${name}')
@operation(summary='Greet someone',
tags=['greetings'],
responses={200: {'description': 'A personalized greeting'}})
async def hello_name(ctx: HttpContext, name: str) -> None:
await ctx.send_str(200, f'Hello {name}')
@app.GET('/square/${x:int}')
@operation(summary='Compute the square of a number', tags=['math'])
async def square(ctx: HttpContext, x: int) -> None:
await ctx.send_str(200, str(x * x))
# serve with an ASGI/RSGI server, e.g.:
# granian --interface rsgi example.openapi:app
# then open http://localhost:8000/docs to browse the API
+10
View File
@@ -145,6 +145,9 @@ class KayaApp(AbstractKayaApp):
await handler.handle_request(ctx, captured)
else:
await ctx.send_empty(404)
except Exception as exc:
ctx.exception = exc
raise
finally:
for hook in reversed(self._after_request_hooks):
await hook(ctx)
@@ -161,6 +164,9 @@ class KayaApp(AbstractKayaApp):
await handler.handle_request(ws, captured)
else:
await ws.close(1000)
except Exception as exc:
ws.exception = exc
raise
finally:
for hook in reversed(self._after_websocket_hooks):
await hook(ws)
@@ -173,6 +179,10 @@ class KayaApp(AbstractKayaApp):
for mixin in self._mixins:
mixin.shutdown(loop)
def route_template(self, url: str, method: HttpMethod = HttpMethod.GET) -> Optional[str]:
"""Return the registered route template that would handle ``url``."""
return self._tree.route_template(url, method)
def route(self,
paths: StrOrStrings,
methods: Optional[HttpMethod | Sequence[HttpMethod]] = None,
@@ -29,6 +29,7 @@ class HttpContext(ABC):
server: Optional[Tuple[str, Optional[int]]]
request_body: AsyncIterator[bytes]
session: Optional[Any] = None
exception: Optional[BaseException] = None
@abstractmethod
async def stream_body(self,
+15 -2
View File
@@ -29,9 +29,22 @@ class KayaMixin(ABC):
pass
def setup(self, loop: AbstractEventLoop) -> None:
"""Called on lifespan startup (default: no-op)."""
"""Called on lifespan startup (default: no-op).
Under RSGI (granian) the loop is NOT yet running when this is
called: schedule work on the passed ``loop`` (e.g.
``loop.create_task``) instead of calling
``asyncio.get_running_loop()``, which raises ``RuntimeError``
there. Under ASGI the lifespan runs inside a coroutine, so a
running loop happens to be available — do not rely on that.
"""
pass
def shutdown(self, loop: AbstractEventLoop) -> None:
"""Called on lifespan shutdown (default: no-op)."""
"""Called on lifespan shutdown (default: no-op).
As with :meth:`setup`, use the passed ``loop``; it may already be
stopped under RSGI, so ``asyncio.get_running_loop()`` is not
reliable here either.
"""
pass
+91 -1
View File
@@ -92,6 +92,16 @@ class Tree:
result = child
key = leaf
while key is not None:
existing = self._find_equivalent_matcher(result, key)
if existing is not None:
result = existing
key = next(it, None)
continue
child = result.children.get(key)
if child is not None:
result = child
key = next(it, None)
continue
new_node = self.parse(key, result)
if isinstance(new_node, Node):
result.children[key] = new_node
@@ -109,6 +119,44 @@ class Tree:
def _supports_method(node: Node | PathMatcher, method: HttpMethod) -> bool:
return None in node.supported_methods or method in node.supported_methods
@staticmethod
def _matcher_identity(leaf: str) -> Optional[Tuple[str, str]]:
start = index_of_with_escape(leaf, '${', '\\', 0)
if start >= 0:
start += 2
end = leaf.index('}', start + 2)
definition = leaf[start:end]
try:
colon = definition.index(':')
except ValueError:
colon = None
if colon is None:
name = definition
kind = 'str'
else:
name = definition[:colon]
kind = definition[colon + 1:]
if kind not in ('str', 'int'):
raise ValueError(f"Unknown kind: '{kind}'")
return (kind, name)
if index_of_with_escape(leaf, '*', '\\', 0) >= 0:
return ('glob', leaf)
return None
def _find_equivalent_matcher(self, node: Node | PathMatcher, leaf: str) -> Optional[PathMatcher]:
identity = self._matcher_identity(leaf)
if identity is None:
return None
kind, key = identity
for existing in node.path_matchers:
if kind == 'str' and isinstance(existing, StrMatcher) and existing.name == key:
return existing
if kind == 'int' and isinstance(existing, IntMatcher) and existing.name == key:
return existing
if kind == 'glob' and isinstance(existing, GlobMatcher) and existing.pattern == key:
return existing
return None
def _check_matcher_conflict(self, node: Node | PathMatcher, method: Optional[HttpMethod]) -> None:
new_is_generic = method is None
for existing in node.path_matchers:
@@ -138,16 +186,27 @@ class Tree:
callback: Callable[[Context, Unpack[Any]], Awaitable[None]],
recursive: bool) -> None:
class Handler(PathHandler):
"""PathHandler created by :meth:`Tree.register`.
The original user callback is exposed through the ``callback``
attribute so that extensions (e.g. ``kaya-openapi``) can inspect
it for metadata such as docstrings or decorator attributes.
"""
callback: Callable[[Context, Unpack[Any]], Awaitable[None]]
async def handle_request(self, ctx: Context, captured: Matches) -> None:
args = Maybe.of_nullable(captured.path).map(lambda it: [it]).or_else([])
await callback(ctx, *args, **captured.kwargs)
await self.callback(ctx, *args, **captured.kwargs)
@property
def recursive(self) -> bool:
return recursive
handler = Handler()
# assigned as an instance attribute (not a class attribute) so that
# the function descriptor protocol does not turn it into a bound method
handler.callback = callback
self.add((p for p in PathIterator(path)), method, handler)
def find_node(self, path: Generator[str, None, None], method: HttpMethod = HttpMethod.GET) \
@@ -178,6 +237,37 @@ class Tree:
# return (handler, unmatched)
return None
def route_template(self, url: str, method: HttpMethod = HttpMethod.GET) -> Optional[str]:
"""Return the registered route template that would handle ``url``.
Static segments are returned verbatim, ``${name}``/``${name:int}``
parameter matchers are reconstructed from the matched nodes, and glob
matchers keep their pattern. Returns ``None`` when no route (including
recursive fallback routes) would handle the request.
"""
result = self.find_node((p for p in PathIterator(urlparse(url).path)), method)
if result is None:
return None
node, captured = result
if len(captured.unmatched_paths) > 0 and not any(handler.recursive for handler in node.handlers):
return None
parts: List[str] = []
current: Optional[Node | PathMatcher] = node
while current is not None:
if isinstance(current, Node):
key = current.key
if not isinstance(key, HttpMethod) and key != '/':
parts.append(str(key))
elif isinstance(current, IntMatcher):
parts.append('${%s:int}' % current.name)
elif isinstance(current, StrMatcher):
parts.append('${%s}' % current.name)
elif isinstance(current, GlobMatcher):
parts.append(current.pattern)
current = current.parent
return '/' + '/'.join(reversed(parts)) if parts else '/'
def parse(self, leaf: str, parent: Optional[Node | PathMatcher]) -> Node | PathMatcher:
start = 0
result = index_of_with_escape(leaf, '${', '\\', 0)
@@ -21,6 +21,7 @@ class WebSocket(ABC):
client: Optional[Tuple[str, int]]
server: Optional[Tuple[str, Optional[int]]]
session: Optional[Any] = None
exception: Optional[BaseException] = None
@abstractmethod
async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
+89
View File
@@ -190,3 +190,92 @@ class AsgiTest(unittest.TestCase):
'employee_id': 101325
}, response)
@async_test
async def test_nested_param_routes(self):
app = KayaApp()
@app.GET('/restaurants/${id}')
async def restaurant(ctx: HttpContext, id: str) -> None:
await ctx.send_str(200, f"restaurant:{id}")
@app.GET('/restaurants/${id}/menu')
async def menu(ctx: HttpContext, id: str) -> None:
await ctx.send_str(200, f"menu:{id}")
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as client:
r = await client.get("/restaurants/42")
self.assertEqual(200, r.status_code)
self.assertEqual("restaurant:42", r.text)
r = await client.get("/restaurants/42/menu")
self.assertEqual(200, r.status_code)
self.assertEqual("menu:42", r.text)
r = await client.get("/restaurants/42/unknown")
self.assertEqual(404, r.status_code)
@async_test
async def test_nested_param_routes_multiple_methods(self):
app = KayaApp()
@app.GET('/restaurants/${id}')
async def restaurant(ctx: HttpContext, id: str) -> None:
await ctx.send_str(200, f"restaurant:{id}")
@app.GET('/restaurants/${id}/menu')
async def menu_get(ctx: HttpContext, id: str) -> None:
await ctx.send_str(200, f"menu_get:{id}")
@app.POST('/restaurants/${id}/menu')
async def menu_post(ctx: HttpContext, id: str) -> None:
await ctx.send_str(200, f"menu_post:{id}")
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as client:
r = await client.get("/restaurants/42")
self.assertEqual(200, r.status_code)
self.assertEqual("restaurant:42", r.text)
r = await client.get("/restaurants/42/menu")
self.assertEqual(200, r.status_code)
self.assertEqual("menu_get:42", r.text)
r = await client.post("/restaurants/42/menu")
self.assertEqual(200, r.status_code)
self.assertEqual("menu_post:42", r.text)
r = await client.put("/restaurants/42/menu")
self.assertEqual(404, r.status_code)
def test_route_template(self):
self.assertEqual('/employee/${employee_id}',
self.app.route_template('/employee/101325', HttpMethod.GET))
self.assertEqual('/square/${x:int}', self.app.route_template('/square/30', HttpMethod.GET))
self.assertIsNone(self.app.route_template('/unknown', HttpMethod.GET))
@async_test
async def test_exception_exposed_to_after_hooks(self):
app = KayaApp()
seen = []
async def after_request(ctx: HttpContext) -> None:
seen.append(ctx.exception)
app.add_after_request_hook(after_request)
@app.GET('/raises')
async def raises(ctx: HttpContext) -> None:
raise RuntimeError('boom')
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as client:
with self.assertRaises(RuntimeError):
await client.get('/raises')
self.assertEqual(1, len(seen))
self.assertIsInstance(seen[0], RuntimeError)
self.assertEqual('boom', str(seen[0]))
+106
View File
@@ -80,11 +80,39 @@ class TreeTest(unittest.TestCase):
self.assertIs(Maybe.of(handler_num).map(self.handlers.__getitem__).or_none(),
Maybe.of_nullable(res).map(lambda it: it[0]).or_none())
def test_route_template(self):
cases: Tuple[Tuple[str, HttpMethod, Optional[str]], ...] = (
('/home/something', HttpMethod.GET, '/home/something'),
('/home/something_else', HttpMethod.POST, '/home/something_else'),
('/home/README.md', HttpMethod.GET, '/home/*.md'),
('/home/something/ciao/blah/README.md', HttpMethod.GET, '/home/something/*/blah/*.md'),
('/home/bar/ciao/blah/README.md', HttpMethod.GET, '/home/bar/*'),
('/unknown', HttpMethod.GET, None),
)
for url, method, expected in cases:
with self.subTest(f'{method} {url}'):
self.assertEqual(expected, self.tree.route_template(url, method))
def test_route_template_with_params(self):
tree = Tree()
tree.add((p for p in ('foo', '${id:int}')), HttpMethod.PUT, self.handlers[0])
tree.add((p for p in ('foo', '${name}')), HttpMethod.GET, self.handlers[1])
self.assertEqual('/foo/${id:int}', tree.route_template('/foo/42', HttpMethod.PUT))
self.assertEqual('/foo/${name}', tree.route_template('/foo/bar', HttpMethod.GET))
self.assertIsNone(tree.route_template('/foo/bar', HttpMethod.DELETE))
def test_two_method_agnostic_matchers_raise(self):
tree = Tree()
tree.add((p for p in ('foo', '*')), None, self.handlers[0])
with self.assertRaises(ValueError):
tree.add((p for p in ('foo', '*.md')), None, self.handlers[1])
def test_identical_method_agnostic_matchers_reuse(self):
tree = Tree()
tree.add((p for p in ('foo', '*')), None, self.handlers[0])
tree.add((p for p in ('foo', '*')), None, self.handlers[1])
handler = Maybe.of_nullable(tree.get_handler('/foo/bar', HttpMethod.GET)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], handler)
def test_two_overlapping_method_specific_matchers_raise(self):
tree = Tree()
@@ -101,3 +129,81 @@ class TreeTest(unittest.TestCase):
self.assertIs(self.handlers[0], put_handler)
self.assertIs(self.handlers[1], get_handler)
def test_nested_routes_with_same_param_allowed(self):
tree = Tree()
tree.add((p for p in ('restaurants', '${id}')), HttpMethod.GET, self.handlers[0])
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.GET, self.handlers[1])
h0 = Maybe.of_nullable(tree.get_handler('/restaurants/42', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h1 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.GET)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], h0)
self.assertIs(self.handlers[1], h1)
def test_nested_routes_same_param_reverse_order(self):
tree = Tree()
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.GET, self.handlers[1])
tree.add((p for p in ('restaurants', '${id}')), HttpMethod.GET, self.handlers[0])
h0 = Maybe.of_nullable(tree.get_handler('/restaurants/42', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h1 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.GET)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], h0)
self.assertIs(self.handlers[1], h1)
def test_nested_routes_with_same_int_param_allowed(self):
tree = Tree()
tree.add((p for p in ('restaurants', '${id:int}')), HttpMethod.GET, self.handlers[0])
tree.add((p for p in ('restaurants', '${id:int}', 'menu')), HttpMethod.GET, self.handlers[1])
h0 = Maybe.of_nullable(tree.get_handler('/restaurants/42', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h1 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.GET)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], h0)
self.assertIs(self.handlers[1], h1)
def test_nested_routes_method_agnostic_reuses_matcher(self):
tree = Tree()
tree.add((p for p in ('restaurants', '${id}')), HttpMethod.GET, self.handlers[0])
tree.add((p for p in ('restaurants', '${id}', 'menu')), None, self.handlers[1])
h0 = Maybe.of_nullable(tree.get_handler('/restaurants/42', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h1 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.POST)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], h0)
self.assertIs(self.handlers[1], h1)
def test_different_param_names_still_raise(self):
tree = Tree()
tree.add((p for p in ('a', '${id}')), HttpMethod.GET, self.handlers[0])
with self.assertRaises(ValueError):
tree.add((p for p in ('a', '${name}', 'x')), HttpMethod.GET, self.handlers[1])
def test_different_param_kinds_still_raise(self):
tree = Tree()
tree.add((p for p in ('a', '${id}')), HttpMethod.GET, self.handlers[0])
with self.assertRaises(ValueError):
tree.add((p for p in ('a', '${id:int}', 'x')), HttpMethod.GET, self.handlers[1])
def test_nested_routes_different_methods_share_subtree(self):
tree = Tree()
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.GET, self.handlers[0])
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.POST, self.handlers[1])
h0 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h1 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.POST)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], h0)
self.assertIs(self.handlers[1], h1)
def test_nested_routes_different_methods_reverse_order(self):
tree = Tree()
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.POST, self.handlers[1])
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.GET, self.handlers[0])
h0 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h1 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.POST)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], h0)
self.assertIs(self.handlers[1], h1)
def test_nested_routes_combined_detail_and_menu_methods(self):
tree = Tree()
tree.add((p for p in ('restaurants', '${id}')), HttpMethod.GET, self.handlers[0])
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.GET, self.handlers[1])
tree.add((p for p in ('restaurants', '${id}', 'menu')), HttpMethod.POST, self.handlers[2])
h0 = Maybe.of_nullable(tree.get_handler('/restaurants/42', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h1 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.GET)).map(lambda it: it[0]).or_none()
h2 = Maybe.of_nullable(tree.get_handler('/restaurants/42/menu', HttpMethod.POST)).map(lambda it: it[0]).or_none()
self.assertIs(self.handlers[0], h0)
self.assertIs(self.handlers[1], h1)
self.assertIs(self.handlers[2], h2)
+63
View File
@@ -0,0 +1,63 @@
# kaya-cors
CORS (Cross-Origin Resource Sharing) support for the Kaya web framework.
Provides `CorsMixin`, a `KayaMixin` that adds CORS response headers to outgoing
responses and answers CORS preflight (`OPTIONS`) requests, with the same
configuration parameters and semantics as FastAPI/Starlette's `CORSMiddleware`.
## Usage
```python
from kaya.core import KayaApp, HttpContext
from kaya.cors import CorsMixin
app = KayaApp(mixins=[
CorsMixin(
allow_origins=['https://example.com'],
allow_methods=('GET', 'POST'),
allow_headers=('X-Custom-Header',),
allow_credentials=True,
max_age=600,
)
])
@app.GET('/')
async def home(ctx: HttpContext):
await ctx.send_str(200, 'Hello World!')
```
## Parameters
- `allow_origins`: list of origins allowed to make cross-origin requests.
Use `['*']` to allow any origin.
- `allow_origin_regex`: optional regex string matched (fullmatch) against the
request origin.
- `allow_methods`: HTTP methods allowed for cross-origin requests
(default `('GET',)`); use `'*'` to allow all standard methods.
- `allow_headers`: request headers allowed in cross-origin requests
(default `()`); use `'*'` to mirror back any requested headers.
- `allow_credentials`: allow cookies/credentials in cross-origin requests
(default `False`). When enabled, the allowed origin is always echoed
explicitly instead of `'*'`.
- `expose_headers`: response headers made accessible to the browser.
- `max_age`: seconds browsers may cache the preflight response
(default `600`).
## Behavior
- Requests without an `Origin` header pass through untouched.
- Simple cross-origin requests with an allowed origin get
`Access-Control-Allow-Origin` (plus `Access-Control-Allow-Credentials` and
`Access-Control-Expose-Headers` when configured) added to the response.
Headers already set by the handler are never overwritten.
- Preflight requests (`OPTIONS` with `Origin` and
`Access-Control-Request-Method` headers) are answered directly by the mixin
with `200 OK` (or `400` with a `Disallowed CORS ...` body when the origin,
method or headers are not allowed). The preflight response is the only one
delivered to the client: if the routing tree matches the request anyway
(including user-registered `OPTIONS` handlers or the 404 fallback), its
output is discarded.
`CorsMixin` is a `KayaMixin`, so the app stays a `KayaApp` and both ASGI and
RSGI keep working.
+56
View File
@@ -0,0 +1,56 @@
[build-system]
requires = ["setuptools>=61.0", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"
[project]
name = "kaya-cors"
dynamic = ["version"]
authors = [
{ name="Walter Oggioni", email="oggioni.walter@gmail.com" },
]
description = "CORS support for the Kaya lightweight ASGI web framework"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
classifiers = [
'Development Status :: 3 - Alpha',
'Topic :: Utilities',
'Intended Audience :: System Administrators',
'Intended Audience :: Developers',
'Environment :: Console',
'Programming Language :: Python :: 3',
]
dependencies = [
"kaya-core",
]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws", "kaya-rsgi"
]
[project.urls]
"Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
[tool.mypy]
python_version = "3.12"
disallow_untyped_defs = true
show_error_codes = true
no_implicit_optional = true
warn_return_any = true
warn_unused_ignores = true
exclude = ["scripts", "docs", "test"]
strict = true
[tool.setuptools_scm]
root = "../.."
version_file = "src/kaya/cors/_version.py"
[tool.setuptools_scm.tag]
prefix = "release/"
@@ -0,0 +1,10 @@
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
from ._mixin import CorsMixin
__all__ = [
'CorsMixin',
]
+274
View File
@@ -0,0 +1,274 @@
import re
from pathlib import Path
from typing import (
Any,
AsyncGenerator,
Dict,
List,
Mapping,
Optional,
Sequence,
Tuple,
)
from kaya.core import HttpContext, HttpMethod, KayaApp, KayaMixin
from kaya.core._types import StrOrStrings
ALL_METHODS: Tuple[str, ...] = ("DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT", "QUERY")
SAFELISTED_HEADERS = frozenset({"Accept", "Accept-Language", "Content-Language", "Content-Type"})
def _first_header(headers: Mapping[str, Sequence[str]], name: str) -> Optional[str]:
values = headers.get(name)
if not values:
return None
return values[0]
def _merge_headers(headers: Optional[Mapping[str, StrOrStrings]],
cors_headers: Mapping[str, str]) -> Mapping[str, StrOrStrings]:
"""Merge CORS headers into the response headers.
Header names are matched case-insensitively; headers already set by the
handler are never overwritten. A CORS ``Vary`` value is appended to an
existing ``Vary`` header when not already present.
"""
result: Dict[str, StrOrStrings] = dict(headers) if headers else {}
key_by_lower: Dict[str, str] = {k.lower(): k for k in result}
for key, value in cors_headers.items():
existing_key = key_by_lower.get(key.lower())
if existing_key is None:
result[key] = value
key_by_lower[key.lower()] = key
elif key.lower() == 'vary':
previous = result[existing_key]
previous_values = [previous] if isinstance(previous, str) else list(previous)
present = {v.strip().lower() for part in previous_values for v in part.split(',')}
if value.lower() not in present:
if isinstance(previous, str):
result[existing_key] = f"{previous}, {value}"
else:
result[existing_key] = (*previous_values, value)
return result
class CorsHttpContext(HttpContext):
"""HttpContext wrapper that injects CORS headers into response headers.
Works with any concrete ``HttpContext`` (ASGI or RSGI) because it only
relies on the abstract send methods, which all implementations share.
Attributes not explicitly overridden are delegated to the wrapped context
via ``__getattr__``, so protocol-specific fields (``pathsend``,
``receive``/``send`` for ASGI, ``protocol`` for RSGI, etc.) are passed
through transparently.
"""
def __init__(self, ctx: HttpContext, cors_headers: Mapping[str, str]) -> None:
object.__setattr__(self, '_ctx', ctx)
object.__setattr__(self, 'session', ctx.session)
object.__setattr__(self, '_cors_headers', cors_headers)
def __getattr__(self, name: str) -> Any:
if name == '_ctx':
raise AttributeError(name)
return getattr(self._ctx, name)
def _merge(self, headers: Optional[Mapping[str, StrOrStrings]]) -> Mapping[str, StrOrStrings]:
return _merge_headers(headers, self._cors_headers)
async def stream_body(self,
status: int,
body_generator: AsyncGenerator[bytes, None],
headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.stream_body(status, body_generator, self._merge(headers))
async def send_bytes(self, status: int, body: bytes, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_bytes(status, body, self._merge(headers))
async def send_str(self, status: int, body: str, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_str(status, body, self._merge(headers))
async def send_file(self, status: int, path: Path, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_file(status, path, self._merge(headers))
async def send_empty(self, status: int, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_empty(status, self._merge(headers))
class _SwallowedHttpContext(HttpContext):
"""HttpContext wrapper whose send methods are no-ops.
Returned by the CORS hook after a preflight response has already been sent,
so that routing (or the 404 fallback) does not attempt to send a second
response for the same request.
"""
def __init__(self, ctx: HttpContext) -> None:
object.__setattr__(self, '_ctx', ctx)
object.__setattr__(self, 'session', ctx.session)
def __getattr__(self, name: str) -> Any:
if name == '_ctx':
raise AttributeError(name)
return getattr(self._ctx, name)
async def stream_body(self,
status: int,
body_generator: AsyncGenerator[bytes, None],
headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
pass
async def send_bytes(self, status: int, body: bytes, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
pass
async def send_file(self, status: int, path: Path, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
pass
async def send_empty(self, status: int, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
pass
class CorsMixin(KayaMixin):
"""Kaya mixin adding CORS headers to responses, modeled after
FastAPI/Starlette's ``CORSMiddleware``.
Registers a before-request hook that:
- answers CORS preflight requests (``OPTIONS`` requests carrying ``Origin``
and ``Access-Control-Request-Method`` headers) directly: ``200 OK`` when
the origin, method and headers are allowed, ``400`` with a
``Disallowed CORS ...`` body otherwise. The preflight response is the
only one delivered to the client; if the routing tree matches the
request anyway, its output is discarded;
- wraps the request context in a :class:`CorsHttpContext` for simple
cross-origin requests, injecting ``Access-Control-Allow-Origin`` (and
the configured credentials/expose headers) into the response.
Requests without an ``Origin`` header pass through untouched. Because the
app stays a ``KayaApp``, both ASGI and RSGI keep working.
Example::
app = KayaApp(mixins=[
CorsMixin(
allow_origins=['https://example.com'],
allow_methods=('GET', 'POST'),
allow_headers=('X-Custom-Header',),
allow_credentials=True,
)
])
"""
def __init__(self,
allow_origins: Sequence[str] = (),
allow_methods: Sequence[str] = ('GET',),
allow_headers: Sequence[str] = (),
allow_credentials: bool = False,
allow_origin_regex: Optional[str] = None,
expose_headers: Sequence[str] = (),
max_age: int = 600) -> None:
methods: Sequence[str] = ALL_METHODS if '*' in allow_methods else allow_methods
self._allow_origin_regex = re.compile(allow_origin_regex) if allow_origin_regex is not None else None
self._allow_all_origins = '*' in allow_origins
self._allow_all_headers = '*' in allow_headers
self._allow_credentials = allow_credentials
self._allow_origins = tuple(allow_origins)
self._allow_methods = tuple(methods)
sorted_allow_headers = sorted(SAFELISTED_HEADERS | set(allow_headers))
self._allow_headers = [h.lower() for h in sorted_allow_headers]
self._preflight_explicit_allow_origin = not self._allow_all_origins or allow_credentials
simple_headers: Dict[str, str] = {}
if self._allow_all_origins:
simple_headers['Access-Control-Allow-Origin'] = '*'
if allow_credentials:
simple_headers['Access-Control-Allow-Credentials'] = 'true'
if expose_headers:
simple_headers['Access-Control-Expose-Headers'] = ', '.join(expose_headers)
self._simple_headers = simple_headers
preflight_headers: Dict[str, str] = {}
if self._preflight_explicit_allow_origin:
# the origin value is set dynamically in _preflight_response()
preflight_headers['Vary'] = 'Origin'
else:
preflight_headers['Access-Control-Allow-Origin'] = '*'
preflight_headers['Access-Control-Allow-Methods'] = ', '.join(self._allow_methods)
preflight_headers['Access-Control-Max-Age'] = str(max_age)
if sorted_allow_headers and not self._allow_all_headers:
preflight_headers['Access-Control-Allow-Headers'] = ', '.join(sorted_allow_headers)
if allow_credentials:
preflight_headers['Access-Control-Allow-Credentials'] = 'true'
self._preflight_headers = preflight_headers
def apply(self, app: KayaApp) -> None:
app.add_before_request_hook(self._before_request)
def _is_allowed_origin(self, origin: str) -> bool:
if self._allow_all_origins:
return True
if self._allow_origin_regex is not None and self._allow_origin_regex.fullmatch(origin):
return True
return origin in self._allow_origins
def _simple_response_headers(self, origin: str) -> Mapping[str, str]:
headers = dict(self._simple_headers)
if self._allow_all_origins and self._allow_credentials:
# credentials require the specific origin instead of '*'
headers['Access-Control-Allow-Origin'] = origin
headers['Vary'] = 'Origin'
elif not self._allow_all_origins and self._is_allowed_origin(origin):
# specific origins must be mirrored back in the response
headers['Access-Control-Allow-Origin'] = origin
headers['Vary'] = 'Origin'
return headers
def _preflight_response(self,
request_headers: Mapping[str, Sequence[str]],
origin: str) -> Tuple[int, str, Mapping[str, str]]:
requested_method = _first_header(request_headers, 'access-control-request-method')
requested_headers = _first_header(request_headers, 'access-control-request-headers')
headers = dict(self._preflight_headers)
failures: List[str] = []
if self._is_allowed_origin(origin):
if self._preflight_explicit_allow_origin:
# the "else" case is already accounted for in self._preflight_headers
# and the value would be '*'
headers['Access-Control-Allow-Origin'] = origin
else:
failures.append('origin')
if requested_method not in self._allow_methods:
failures.append('method')
# if we allow all headers, then we have to mirror back any requested
# headers in the response
if self._allow_all_headers and requested_headers is not None:
headers['Access-Control-Allow-Headers'] = requested_headers
elif requested_headers is not None:
for header in [h.lower() for h in requested_headers.split(',')]:
if header.strip() not in self._allow_headers:
failures.append('headers')
break
# we don't strictly need to use 400 responses here, since it's up to
# the browser to enforce the CORS policy, but it's more informative
# if we do
if failures:
return 400, 'Disallowed CORS ' + ', '.join(failures), headers
return 200, 'OK', headers
async def _before_request(self, ctx: HttpContext) -> Optional[HttpContext]:
origin = _first_header(ctx.headers, 'origin')
if origin is None:
return None
if ctx.method == HttpMethod.OPTIONS \
and _first_header(ctx.headers, 'access-control-request-method') is not None:
status, body, headers = self._preflight_response(ctx.headers, origin)
response_headers: Dict[str, StrOrStrings] = {'Content-Type': 'text/plain; charset=utf-8'}
response_headers.update(headers)
await ctx.send_str(status, body, response_headers)
return _SwallowedHttpContext(ctx)
return CorsHttpContext(ctx, self._simple_response_headers(origin))
+246
View File
@@ -0,0 +1,246 @@
import asyncio
import unittest
from typing import Any, Optional
import httpx
from pwo import async_test
from kaya.core import HttpContext, KayaApp
from kaya.cors import CorsMixin
def make_app(**cors_kwargs: Any) -> KayaApp:
app = KayaApp(mixins=[CorsMixin(**cors_kwargs)])
@app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
await ctx.send_str(200, 'Hello World!')
return app
async def request(app: KayaApp,
method: str,
path: str = '/hello',
headers: Optional[dict[str, str]] = None) -> httpx.Response:
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as client:
return await client.request(method, path, headers=headers)
class CorsSimpleRequestTest(unittest.TestCase):
@async_test
async def test_allowed_origin(self):
app = make_app(allow_origins=['https://example.com'])
r = await request(app, 'GET', headers={'Origin': 'https://example.com'})
self.assertEqual(200, r.status_code)
self.assertEqual('Hello World!', r.text)
self.assertEqual('https://example.com', r.headers.get('Access-Control-Allow-Origin'))
self.assertEqual('Origin', r.headers.get('Vary'))
@async_test
async def test_disallowed_origin(self):
app = make_app(allow_origins=['https://example.com'])
r = await request(app, 'GET', headers={'Origin': 'https://evil.com'})
self.assertEqual(200, r.status_code)
self.assertNotIn('Access-Control-Allow-Origin', r.headers)
@async_test
async def test_wildcard_origin(self):
app = make_app(allow_origins=['*'])
r = await request(app, 'GET', headers={'Origin': 'https://anything.example.com'})
self.assertEqual('*', r.headers.get('Access-Control-Allow-Origin'))
@async_test
async def test_wildcard_origin_with_credentials_echoes_origin(self):
app = make_app(allow_origins=['*'], allow_credentials=True)
r = await request(app, 'GET', headers={'Origin': 'https://example.com'})
self.assertEqual('https://example.com', r.headers.get('Access-Control-Allow-Origin'))
self.assertEqual('true', r.headers.get('Access-Control-Allow-Credentials'))
self.assertEqual('Origin', r.headers.get('Vary'))
@async_test
async def test_origin_regex(self):
app = make_app(allow_origin_regex=r'https://.*\.example\.com')
r = await request(app, 'GET', headers={'Origin': 'https://api.example.com'})
self.assertEqual('https://api.example.com', r.headers.get('Access-Control-Allow-Origin'))
r = await request(app, 'GET', headers={'Origin': 'https://example.com.evil.org'})
self.assertNotIn('Access-Control-Allow-Origin', r.headers)
@async_test
async def test_no_origin_header(self):
app = make_app(allow_origins=['*'])
r = await request(app, 'GET')
self.assertEqual(200, r.status_code)
self.assertNotIn('Access-Control-Allow-Origin', r.headers)
@async_test
async def test_expose_headers(self):
app = make_app(allow_origins=['*'], expose_headers=['X-Total-Count'])
r = await request(app, 'GET', headers={'Origin': 'https://example.com'})
self.assertEqual('X-Total-Count', r.headers.get('Access-Control-Expose-Headers'))
@async_test
async def test_handler_set_cors_header_not_overwritten(self):
app = KayaApp(mixins=[CorsMixin(allow_origins=['*'])])
@app.GET('/custom')
async def custom(ctx: HttpContext) -> None:
await ctx.send_str(200, 'custom', headers={'Access-Control-Allow-Origin': 'https://custom.example.com'})
r = await request(app, 'GET', '/custom', headers={'Origin': 'https://example.com'})
self.assertEqual('https://custom.example.com', r.headers.get('Access-Control-Allow-Origin'))
class CorsPreflightTest(unittest.TestCase):
@staticmethod
def preflight_headers(origin: str = 'https://example.com',
method: str = 'POST',
headers: Optional[str] = None) -> dict[str, str]:
result = {
'Origin': origin,
'Access-Control-Request-Method': method,
}
if headers is not None:
result['Access-Control-Request-Headers'] = headers
return result
@async_test
async def test_preflight_allowed(self):
app = make_app(allow_origins=['https://example.com'],
allow_methods=('GET', 'POST'),
allow_credentials=True)
r = await request(app, 'OPTIONS', headers=self.preflight_headers())
self.assertEqual(200, r.status_code)
self.assertEqual('OK', r.text)
self.assertEqual('https://example.com', r.headers.get('Access-Control-Allow-Origin'))
self.assertEqual('GET, POST', r.headers.get('Access-Control-Allow-Methods'))
self.assertEqual('600', r.headers.get('Access-Control-Max-Age'))
self.assertEqual('true', r.headers.get('Access-Control-Allow-Credentials'))
self.assertEqual('Origin', r.headers.get('Vary'))
@async_test
async def test_preflight_wildcard_origin(self):
app = make_app(allow_origins=['*'], allow_methods=('GET', 'POST'))
r = await request(app, 'OPTIONS', headers=self.preflight_headers())
self.assertEqual(200, r.status_code)
self.assertEqual('*', r.headers.get('Access-Control-Allow-Origin'))
@async_test
async def test_preflight_disallowed_origin(self):
app = make_app(allow_origins=['https://example.com'], allow_methods=('GET', 'POST'))
r = await request(app, 'OPTIONS', headers=self.preflight_headers(origin='https://evil.com'))
self.assertEqual(400, r.status_code)
self.assertEqual('Disallowed CORS origin', r.text)
@async_test
async def test_preflight_disallowed_method(self):
app = make_app(allow_origins=['https://example.com'], allow_methods=('GET',))
r = await request(app, 'OPTIONS', headers=self.preflight_headers(method='DELETE'))
self.assertEqual(400, r.status_code)
self.assertEqual('Disallowed CORS method', r.text)
@async_test
async def test_preflight_disallowed_headers(self):
app = make_app(allow_origins=['https://example.com'], allow_methods=('GET', 'POST'))
r = await request(app, 'OPTIONS', headers=self.preflight_headers(headers='X-Custom'))
self.assertEqual(400, r.status_code)
self.assertEqual('Disallowed CORS headers', r.text)
@async_test
async def test_preflight_safelisted_headers_allowed(self):
app = make_app(allow_origins=['https://example.com'], allow_methods=('GET', 'POST'))
r = await request(app, 'OPTIONS', headers=self.preflight_headers(headers='Content-Type'))
self.assertEqual(200, r.status_code)
@async_test
async def test_preflight_allow_all_headers_mirrors_request(self):
app = make_app(allow_origins=['*'], allow_methods=('GET', 'POST'), allow_headers=['*'])
r = await request(app, 'OPTIONS', headers=self.preflight_headers(headers='X-Custom, X-Other'))
self.assertEqual(200, r.status_code)
self.assertEqual('X-Custom, X-Other', r.headers.get('Access-Control-Allow-Headers'))
@async_test
async def test_preflight_configured_allow_headers(self):
app = make_app(allow_origins=['https://example.com'],
allow_methods=('GET', 'POST'),
allow_headers=('X-Custom',))
r = await request(app, 'OPTIONS', headers=self.preflight_headers(headers='X-Custom'))
self.assertEqual(200, r.status_code)
allow_headers = r.headers.get('Access-Control-Allow-Headers')
self.assertIsNotNone(allow_headers)
assert allow_headers is not None
self.assertIn('X-Custom', allow_headers)
@async_test
async def test_preflight_response_not_overwritten_by_handler(self):
# even when a user-registered OPTIONS handler matches, the preflight
# response sent by the mixin is the only one delivered to the client
app = KayaApp(mixins=[CorsMixin(allow_origins=['https://example.com'], allow_methods=('GET', 'POST'))])
@app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
await ctx.send_str(200, 'Hello World!')
@app.OPTIONS('/hello')
async def options(ctx: HttpContext) -> None:
await ctx.send_str(200, 'custom OPTIONS handler')
r = await request(app, 'OPTIONS', headers=self.preflight_headers())
self.assertEqual(200, r.status_code)
self.assertEqual('OK', r.text)
@async_test
async def test_options_without_preflight_headers_routes_normally(self):
app = KayaApp(mixins=[CorsMixin(allow_origins=['https://example.com'])])
@app.OPTIONS('/hello')
async def options(ctx: HttpContext) -> None:
await ctx.send_str(200, 'custom OPTIONS handler')
# an OPTIONS request without Access-Control-Request-Method is not a
# preflight request and is routed normally
r = await request(app, 'OPTIONS', headers={'Origin': 'https://example.com'})
self.assertEqual(200, r.status_code)
self.assertEqual('custom OPTIONS handler', r.text)
self.assertEqual('https://example.com', r.headers.get('Access-Control-Allow-Origin'))
class CorsRsgiTest(unittest.TestCase):
def test_rsgi_context_header_injection(self):
from kaya.rsgi import RsgiContext
class FakeScope:
scheme = 'http'
method = 'GET'
path = '/'
query_string = ''
headers = {'origin': 'https://example.com'}
client = '127.0.0.1:12345'
server = '127.0.0.1:80'
class FakeProtocol:
def __init__(self) -> None:
self.responses = []
def response_str(self, status: int, headers: list, body: str) -> None:
self.responses.append((status, dict(headers), body))
mixin = CorsMixin(allow_origins=['https://example.com'])
protocol = FakeProtocol()
ctx = RsgiContext(FakeScope(), protocol) # type: ignore[arg-type]
async def run() -> None:
wrapped = await mixin._before_request(ctx)
assert wrapped is not None
await wrapped.send_str(200, 'hi')
asyncio.run(run())
self.assertEqual(1, len(protocol.responses))
status, headers, body = protocol.responses[0]
self.assertEqual(200, status)
self.assertEqual('https://example.com', headers.get('Access-Control-Allow-Origin'))
self.assertEqual('Origin', headers.get('Vary'))
+61
View File
@@ -0,0 +1,61 @@
# kaya-forwarded
Trusted-proxy forwarded header handling for the Kaya web framework.
Without this package, Kaya exposes the raw socket peer address as
`ctx.client` / `ws.client` and ignores `Forwarded` / `X-Forwarded-*` headers
entirely (they are client-controllable and trivially spoofable when the app is
directly exposed).
`ForwardedHeadersMixin` opts the application into honoring those headers, but
only when the direct socket peer is a trusted proxy, identified by a list of
trusted CIDRs/IPs.
## Usage
```python
from kaya.core import KayaApp, HttpContext
from kaya.forwarded import ForwardedHeadersMixin
app = KayaApp(mixins=[
ForwardedHeadersMixin(trusted_proxies=['127.0.0.1', '::1', '10.0.0.0/8'])
])
@app.GET('/whoami')
async def whoami(ctx: HttpContext):
host, port = ctx.client
await ctx.send_str(200, f'{host}:{port}')
```
## How it works
When a request arrives:
1. If the socket peer IP does not belong to any trusted CIDR (or there is no
peer address), the mixin leaves the context untouched — `client` remains
the socket peer and all proxy headers are ignored.
2. Otherwise the client address is resolved from the headers, in order:
- `Forwarded` (RFC 7239): the `for=` entries are walked **from right to
left**, skipping entries that are themselves trusted proxies (and
`unknown`); the first untrusted entry is the client. This defeats
spoofing when the edge proxy *appends* to the header (e.g. nginx with
`$proxy_add_x_forwarded_for`), because attacker-supplied leftmost entries
are never selected. A `:port` in the selected `for=` value also
populates the port.
- `X-Forwarded-For`: same right-to-left trusted-proxy walk; the port comes
from `X-Forwarded-Port` when present and valid.
- `X-Forwarded-Host`: first entry; port from `X-Forwarded-Port` as above.
3. If none of the headers are present or usable, the socket peer is kept.
If every entry in the chain is a trusted proxy, the leftmost entry is used
(the whole chain is trusted, so the leftmost is the original client).
The resolved address is exposed by wrapping the request context /
websocket (the same pattern as `kaya-session`), so both ASGI and RSGI keep
working and `ctx.session` from other mixins is preserved.
## Note
Even with this mixin, the edge proxy should still strip or overwrite inbound
`Forwarded` / `X-Forwarded-*` headers from clients — the mixin protects the
application, the proxy protects the chain.
+56
View File
@@ -0,0 +1,56 @@
[build-system]
requires = ["setuptools>=61.0", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"
[project]
name = "kaya-forwarded"
dynamic = ["version"]
authors = [
{ name="Walter Oggioni", email="oggioni.walter@gmail.com" },
]
description = "Trusted-proxy forwarded header handling for the Kaya lightweight ASGI web framework"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
classifiers = [
'Development Status :: 3 - Alpha',
'Topic :: Utilities',
'Intended Audience :: System Administrators',
'Intended Audience :: Developers',
'Environment :: Console',
'Programming Language :: Python :: 3',
]
dependencies = [
"kaya-core",
]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws", "kaya-rsgi"
]
[project.urls]
"Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
[tool.mypy]
python_version = "3.12"
disallow_untyped_defs = true
show_error_codes = true
no_implicit_optional = true
warn_return_any = true
warn_unused_ignores = true
exclude = ["scripts", "docs", "test"]
strict = true
[tool.setuptools_scm]
root = "../.."
version_file = "src/kaya/forwarded/_version.py"
[tool.setuptools_scm.tag]
prefix = "release/"
@@ -0,0 +1,10 @@
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
from ._mixin import ForwardedHeadersMixin
__all__ = [
'ForwardedHeadersMixin',
]
@@ -0,0 +1,237 @@
from ipaddress import ip_address, ip_network, IPv4Address, IPv4Network, IPv6Address, IPv6Network
from pathlib import Path
from typing import (
Any,
AsyncGenerator,
List,
Mapping,
Optional,
Sequence,
Tuple,
Union,
)
from kaya.core import HttpContext, KayaApp, KayaMixin, WebSocket, WebSocketMessage
from kaya.core._types import StrOrStrings
_IPAddress = Union[IPv4Address, IPv6Address]
_IPNetwork = Union[IPv4Network, IPv6Network]
def _split_host_port(value: str) -> Tuple[str, Optional[int]]:
if value.startswith('['):
# bracketed IPv6 address, optionally followed by :port
closing = value.find(']')
if closing == -1:
return value, None
host = value[1:closing]
rest = value[closing + 1:]
if rest.startswith(':'):
try:
return host, int(rest[1:])
except ValueError:
return host, None
return host, None
if value.count(':') == 1:
host, _, port_str = value.rpartition(':')
try:
return host, int(port_str)
except ValueError:
return value, None
return value, None
def _parse_forwarded_entries(headers: Mapping[str, Sequence[str]]) -> List[Tuple[str, Optional[int]]]:
"""Extract the (host, port) `for=` entries of the RFC 7239 `Forwarded` header,
flattened across all header values, in chain order (leftmost = original client).
"""
entries: List[Tuple[str, Optional[int]]] = []
for raw_value in headers.get('forwarded', ()):
for element in raw_value.split(','):
for param in element.split(';'):
key, sep, value = param.partition('=')
if sep and key.strip().lower() == 'for':
for_value = value.strip().strip('"')
if for_value and for_value.lower() != 'unknown':
entries.append(_split_host_port(for_value))
break
return entries
def _first_header_value(headers: Mapping[str, Sequence[str]], name: str) -> Optional[str]:
values = headers.get(name)
if not values:
return None
first = values[0].split(',')[0].strip()
return first or None
class _ForwardedHttpContext(HttpContext):
"""HttpContext wrapper that exposes the forwarded client address.
Everything except ``client`` is delegated to the wrapped context via
``__getattr__``, so it works with any concrete ``HttpContext`` (ASGI or
RSGI) and preserves attributes set by other mixins (e.g. ``session``).
"""
def __init__(self, ctx: HttpContext, client: Tuple[str, int]) -> None:
object.__setattr__(self, '_ctx', ctx)
object.__setattr__(self, 'session', ctx.session)
object.__setattr__(self, 'client', client)
def __getattr__(self, name: str) -> Any:
if name == '_ctx':
raise AttributeError(name)
return getattr(self._ctx, name)
async def stream_body(self,
status: int,
body_generator: AsyncGenerator[bytes, None],
headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.stream_body(status, body_generator, headers)
async def send_bytes(self, status: int, body: bytes, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_bytes(status, body, headers)
async def send_str(self, status: int, body: str, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_str(status, body, headers)
async def send_file(self, status: int, path: Path, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_file(status, path, headers)
async def send_empty(self, status: int, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ctx.send_empty(status, headers)
class _ForwardedWebSocket(WebSocket):
"""WebSocket wrapper that exposes the forwarded client address.
Everything except ``client`` is delegated to the wrapped socket via
``__getattr__``, so it works with any concrete ``WebSocket`` (ASGI or
RSGI) and preserves attributes set by other mixins (e.g. ``session``).
"""
def __init__(self, ws: WebSocket, client: Tuple[str, int]) -> None:
object.__setattr__(self, '_ws', ws)
object.__setattr__(self, 'session', ws.session)
object.__setattr__(self, 'client', client)
def __getattr__(self, name: str) -> Any:
if name == '_ws':
raise AttributeError(name)
return getattr(self._ws, name)
async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ws.accept(headers)
async def receive(self) -> WebSocketMessage:
return await self._ws.receive()
async def send_text(self, data: str) -> None:
await self._ws.send_text(data)
async def send_bytes(self, data: bytes) -> None:
await self._ws.send_bytes(data)
async def close(self, code: int = 1000) -> None:
await self._ws.close(code)
async def __anext__(self) -> WebSocketMessage:
return await self._ws.__anext__()
class ForwardedHeadersMixin(KayaMixin):
"""Kaya mixin that resolves the client address from proxy headers
(``Forwarded``, ``X-Forwarded-For``, ``X-Forwarded-Host``), but only when
the direct socket peer is a trusted proxy.
Without this mixin, Kaya exposes the raw socket peer as ``ctx.client`` /
``ws.client`` and ignores forwarded headers entirely. With the mixin
applied, forwarded headers are honored only if the socket peer IP belongs
to one of the ``trusted_proxies`` CIDRs; otherwise the context is left
untouched.
When the peer is trusted, the address chain is walked from right to left
and entries that are themselves trusted proxies are skipped, so a client
that prepends a spoofed entry cannot fool the resolution when the edge
proxy appends to the header (e.g. nginx with ``$proxy_add_x_forwarded_for``).
Example::
app = KayaApp(mixins=[
ForwardedHeadersMixin(trusted_proxies=['127.0.0.1', '::1', '10.0.0.0/8'])
])
"""
def __init__(self, trusted_proxies: Sequence[str] = ()) -> None:
self._trusted_networks: Tuple[_IPNetwork, ...] = tuple(
ip_network(cidr, strict=False) for cidr in trusted_proxies
)
def apply(self, app: KayaApp) -> None:
app.add_before_request_hook(self._before_request)
app.add_before_websocket_hook(self._before_websocket)
def _is_trusted(self, host: str) -> bool:
try:
addr: _IPAddress = ip_address(host)
except ValueError:
return False
return any(addr.version == network.version and addr in network
for network in self._trusted_networks)
def _select_forwarded_entry(self, entries: List[Tuple[str, Optional[int]]]) -> Optional[Tuple[str, Optional[int]]]:
"""Walk the chain right-to-left skipping trusted proxies; the first
untrusted entry is the client. If every entry is trusted, the leftmost
(original client) is returned.
"""
for entry in reversed(entries):
if not self._is_trusted(entry[0]):
return entry
return entries[0] if entries else None
def _forwarded_port(self, headers: Mapping[str, Sequence[str]], socket_port: int) -> int:
forwarded_port = _first_header_value(headers, 'x-forwarded-port')
if forwarded_port is not None:
try:
return int(forwarded_port)
except ValueError:
pass
return socket_port
def _resolve(self,
headers: Mapping[str, Sequence[str]],
client: Optional[Tuple[str, int]]) -> Optional[Tuple[str, int]]:
if client is None or not self._is_trusted(client[0]):
return client
socket_port = client[1]
selected = self._select_forwarded_entry(_parse_forwarded_entries(headers))
if selected is not None:
host, port = selected
return host, port if port is not None else socket_port
xff_values = headers.get('x-forwarded-for')
if xff_values:
xff_entries = [entry.strip() for value in xff_values for entry in value.split(',') if entry.strip()]
selected_host = self._select_forwarded_entry([(entry, None) for entry in xff_entries])
if selected_host is not None:
return selected_host[0], self._forwarded_port(headers, socket_port)
xfh = _first_header_value(headers, 'x-forwarded-host')
if xfh is not None:
return xfh, self._forwarded_port(headers, socket_port)
return client
async def _before_request(self, ctx: HttpContext) -> Optional[HttpContext]:
resolved = self._resolve(ctx.headers, ctx.client)
if resolved is None or resolved is ctx.client:
return None
return _ForwardedHttpContext(ctx, resolved)
async def _before_websocket(self, ws: WebSocket) -> Optional[WebSocket]:
resolved = self._resolve(ws.headers, ws.client)
if resolved is None or resolved is ws.client:
return None
return _ForwardedWebSocket(ws, resolved)
@@ -0,0 +1,244 @@
import asyncio
import json
import unittest
from typing import Optional
import httpx
from pwo import async_test
from kaya.core import HttpContext, KayaApp
from kaya.core._asgi import AsgiWebSocket
from kaya.forwarded import ForwardedHeadersMixin
TRUSTED = ['127.0.0.1', '::1', '10.0.0.0/8']
def make_app(trusted_proxies=TRUSTED) -> KayaApp:
mixins = [ForwardedHeadersMixin(trusted_proxies=trusted_proxies)] if trusted_proxies is not None else []
app = KayaApp(mixins=mixins)
@app.GET('/client')
async def client(ctx: HttpContext) -> None:
host, port = ctx.client if ctx.client is not None else (None, None)
await ctx.send_str(200, json.dumps({'host': host, 'port': port}))
return app
async def request(app: KayaApp,
headers: Optional[dict[str, str]] = None,
client: tuple[str, int] = ('127.0.0.1', 123)) -> dict:
transport = httpx.ASGITransport(app=app, client=client)
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as http_client:
r = await http_client.get('/client', headers=headers)
assert r.status_code == 200
return json.loads(r.text)
class TrustedPeerTest(unittest.TestCase):
@async_test
async def test_no_headers_returns_socket_peer(self):
app = make_app()
result = await request(app)
self.assertEqual({'host': '127.0.0.1', 'port': 123}, result)
@async_test
async def test_forwarded_header_with_port(self):
app = make_app()
result = await request(app, headers={'Forwarded': 'for=203.0.113.5:1234'})
self.assertEqual({'host': '203.0.113.5', 'port': 1234}, result)
@async_test
async def test_forwarded_header_bracketed_ipv6(self):
app = make_app()
result = await request(app, headers={'Forwarded': 'for="[2001:db8::1]:4711"'})
self.assertEqual({'host': '2001:db8::1', 'port': 4711}, result)
@async_test
async def test_forwarded_header_without_port_keeps_socket_port(self):
app = make_app()
result = await request(app, headers={'Forwarded': 'for=203.0.113.5'})
self.assertEqual({'host': '203.0.113.5', 'port': 123}, result)
@async_test
async def test_forwarded_unknown_entry_skipped(self):
app = make_app()
result = await request(app, headers={'Forwarded': 'for=unknown, for=203.0.113.5'})
self.assertEqual('203.0.113.5', result['host'])
@async_test
async def test_forwarded_rightmost_untrusted_wins(self):
# attacker-controlled leftmost entry is skipped: the rightmost
# untrusted entry (appended by the trusted edge proxy) is the client
app = make_app()
result = await request(app, headers={'Forwarded': 'for=1.2.3.4, for=5.6.7.8, for=10.0.0.2'})
self.assertEqual('5.6.7.8', result['host'])
@async_test
async def test_x_forwarded_for_spoofed_leftmost_entry_skipped(self):
# XFF = "<attacker-supplied>, <real client>" as appended by the proxy
app = make_app()
result = await request(app, headers={'X-Forwarded-For': '1.2.3.4, 5.6.7.8'})
self.assertEqual('5.6.7.8', result['host'])
@async_test
async def test_x_forwarded_for_all_trusted_chain_uses_leftmost(self):
app = make_app()
result = await request(app, headers={'X-Forwarded-For': '10.0.0.5, 10.0.0.2'})
self.assertEqual('10.0.0.5', result['host'])
@async_test
async def test_x_forwarded_port(self):
app = make_app()
result = await request(app, headers={
'X-Forwarded-For': '203.0.113.5',
'X-Forwarded-Port': '8443',
})
self.assertEqual({'host': '203.0.113.5', 'port': 8443}, result)
@async_test
async def test_invalid_x_forwarded_port_ignored(self):
app = make_app()
result = await request(app, headers={
'X-Forwarded-For': '203.0.113.5',
'X-Forwarded-Port': 'not-a-port',
})
self.assertEqual({'host': '203.0.113.5', 'port': 123}, result)
@async_test
async def test_forwarded_takes_precedence_over_x_forwarded_for(self):
app = make_app()
result = await request(app, headers={
'Forwarded': 'for=203.0.113.5',
'X-Forwarded-For': '198.51.100.7',
})
self.assertEqual('203.0.113.5', result['host'])
@async_test
async def test_x_forwarded_host_fallback(self):
app = make_app()
result = await request(app, headers={'X-Forwarded-Host': '198.51.100.7'})
self.assertEqual('198.51.100.7', result['host'])
@async_test
async def test_ipv6_cidr_trust(self):
app = make_app(trusted_proxies=['2001:db8::/32'])
result = await request(app,
headers={'X-Forwarded-For': '203.0.113.5'},
client=('2001:db8::10', 9999))
self.assertEqual({'host': '203.0.113.5', 'port': 9999}, result)
class UntrustedPeerTest(unittest.TestCase):
@async_test
async def test_untrusted_peer_ignores_forwarded_headers(self):
app = make_app()
result = await request(app,
headers={'Forwarded': 'for=1.2.3.4', 'X-Forwarded-For': '1.2.3.4'},
client=('203.0.113.99', 4567))
self.assertEqual({'host': '203.0.113.99', 'port': 4567}, result)
@async_test
async def test_empty_trusted_proxies_ignores_everything(self):
app = make_app(trusted_proxies=[])
result = await request(app, headers={'X-Forwarded-For': '1.2.3.4'})
self.assertEqual({'host': '127.0.0.1', 'port': 123}, result)
@async_test
async def test_invalid_cidr_fails_fast(self):
with self.assertRaises(ValueError):
ForwardedHeadersMixin(trusted_proxies=['not-a-cidr'])
class OptOutTest(unittest.TestCase):
@async_test
async def test_without_mixin_headers_are_ignored(self):
app = KayaApp()
@app.GET('/client')
async def client(ctx: HttpContext) -> None:
host, port = ctx.client if ctx.client is not None else (None, None)
await ctx.send_str(200, json.dumps({'host': host, 'port': port}))
result = await request(app, headers={'Forwarded': 'for=1.2.3.4', 'X-Forwarded-For': '1.2.3.4'})
self.assertEqual({'host': '127.0.0.1', 'port': 123}, result)
class WebSocketTest(unittest.TestCase):
@staticmethod
def _make_ws(headers):
async def send(message):
pass
async def receive():
return {'type': 'websocket.connect'}
scope = {
'type': 'websocket',
'path': '/ws',
'query_string': b'',
'scheme': 'ws',
'client': ('127.0.0.1', 12345),
'server': ('127.0.0.1', 80),
'headers': headers,
}
return AsgiWebSocket(scope, receive, send)
@async_test
async def test_websocket_trusted_peer(self):
mixin = ForwardedHeadersMixin(trusted_proxies=TRUSTED)
ws = self._make_ws([(b'x-forwarded-for', b'1.2.3.4, 5.6.7.8')])
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
self.assertEqual(('5.6.7.8', 12345), wrapped.client)
@async_test
async def test_websocket_untrusted_peer(self):
mixin = ForwardedHeadersMixin(trusted_proxies=['10.0.0.0/8'])
ws = self._make_ws([(b'x-forwarded-for', b'1.2.3.4')])
wrapped = await mixin._before_websocket(ws)
self.assertIsNone(wrapped)
self.assertEqual(('127.0.0.1', 12345), ws.client)
class RsgiTest(unittest.TestCase):
def test_rsgi_context(self):
from kaya.rsgi import RsgiContext
class FakeScope:
scheme = 'http'
method = 'GET'
path = '/'
query_string = ''
headers = {'x-forwarded-for': '1.2.3.4, 5.6.7.8'}
client = '127.0.0.1:12345'
server = '127.0.0.1:80'
mixin = ForwardedHeadersMixin(trusted_proxies=TRUSTED)
ctx = RsgiContext(FakeScope(), object()) # type: ignore[arg-type]
wrapped = asyncio.run(mixin._before_request(ctx))
assert wrapped is not None
self.assertEqual(('5.6.7.8', 12345), wrapped.client)
def test_rsgi_context_untrusted_peer(self):
from kaya.rsgi import RsgiContext
class FakeScope:
scheme = 'http'
method = 'GET'
path = '/'
query_string = ''
headers = {'x-forwarded-for': '1.2.3.4'}
client = '192.0.2.1:12345'
server = '127.0.0.1:80'
mixin = ForwardedHeadersMixin(trusted_proxies=TRUSTED)
ctx = RsgiContext(FakeScope(), object()) # type: ignore[arg-type]
wrapped = asyncio.run(mixin._before_request(ctx))
self.assertIsNone(wrapped)
self.assertEqual(('192.0.2.1', 12345), ctx.client)
+93
View File
@@ -0,0 +1,93 @@
# kaya-openapi
Automatic [OpenAPI](https://www.openapis.org/) specification generation for the
[Kaya](https://github.com/woggioni/kaya) lightweight ASGI web framework.
The package provides an `OpenAPIMixin` that inspects a `KayaApp`'s routing
tree and serves:
- an OpenAPI 3.1 JSON document (default: `GET /openapi.json`)
- a Swagger UI page to browse it interactively (default: `GET /docs`)
## Usage
```python
from kaya.core import HttpContext, KayaApp
from kaya.openapi import OpenAPIMixin, operation
app = KayaApp(mixins=[OpenAPIMixin(title='My API', version='1.0.0')])
@app.GET('/users/${user_id:int}')
@operation(summary='Get a user',
tags=['users'],
responses={
200: {'description': 'The user'},
404: {'description': 'User not found'},
})
async def get_user(ctx: HttpContext, user_id: int) -> None:
...
```
Run the app with any ASGI/RSGI server and open `http://localhost:8000/docs`.
## How routes are mapped
- Static segments and parameters are converted to OpenAPI path templating:
- `/users/${user_id}` &rarr; `/users/{user_id}` (string path parameter)
- `/users/${user_id:int}` &rarr; `/users/{user_id}` (integer path parameter)
- Wildcard routes (`*`) are **skipped**: they cannot be expressed in OpenAPI
path syntax.
- Websocket routes are **skipped**: OpenAPI does not model websockets.
- Method-agnostic routes (registered with `app.route(path)` without methods)
are documented under **all** standard HTTP methods, since they respond to
all of them.
- The mixin's own endpoints are excluded from the document unless
`include_self=True`.
The document is generated on every request to the spec endpoint, so routes
registered after the mixin is applied are always included.
## Operation metadata
The `@operation` decorator attaches OpenAPI metadata to a route handler.
All fragments are plain dicts merged verbatim into the generated operation
object, so any valid OpenAPI 3.1 construct can be used:
```python
@operation(summary='...', # operation summary
description='...', # defaults to the handler docstring
tags=['users'],
operation_id='getUser',
request_body={...}, # OpenAPI requestBody object
responses={200: {...}}, # per-status-code response objects
parameters=[...], # extra/overriding parameter objects
deprecated=False,
hidden=False) # exclude from the document
```
`parameters` entries whose `name` and `in` match an auto-generated path
parameter override it; all others are appended.
## Configuration
```python
OpenAPIMixin(
title='My API', # info.title (required)
version='1.0.0', # info.version (required)
description='', # info.description
spec_path='/openapi.json', # where the JSON document is served
docs_path='/docs', # where Swagger UI is served
servers=[{'url': 'https://api.example.com'}],
openapi_version='3.1.0',
include_self=False, # include spec/docs endpoints in the document
)
```
The document can also be generated programmatically without serving it:
```python
from kaya.openapi import generate_spec
spec = generate_spec(app, title='My API', version='1.0.0')
```
+56
View File
@@ -0,0 +1,56 @@
[build-system]
requires = ["setuptools>=61.0", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"
[project]
name = "kaya-openapi"
dynamic = ["version"]
authors = [
{ name="Walter Oggioni", email="oggioni.walter@gmail.com" },
]
description = "Automatic OpenAPI specification generation for the Kaya lightweight ASGI web framework"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
classifiers = [
'Development Status :: 3 - Alpha',
'Topic :: Utilities',
'Intended Audience :: System Administrators',
'Intended Audience :: Developers',
'Environment :: Console',
'Programming Language :: Python :: 3',
]
dependencies = [
"kaya-core",
]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws"
]
[project.urls]
"Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
[tool.mypy]
python_version = "3.12"
disallow_untyped_defs = true
show_error_codes = true
no_implicit_optional = true
warn_return_any = true
warn_unused_ignores = true
exclude = ["scripts", "docs", "test"]
strict = true
[tool.setuptools_scm]
root = "../.."
version_file = "src/kaya/openapi/_version.py"
[tool.setuptools_scm.tag]
prefix = "release/"
@@ -0,0 +1,10 @@
from ._metadata import operation
from ._mixin import OpenAPIMixin
from ._spec import generate_spec
__all__ = [
'OpenAPIMixin',
'generate_spec',
'operation',
]
@@ -0,0 +1,84 @@
from typing import Any, Awaitable, Callable, Mapping, Optional, Sequence, TypeVar
F = TypeVar('F', bound=Callable[..., Awaitable[None]])
METADATA_ATTR = '__kaya_openapi__'
#: Metadata attached to route handler functions by :func:`operation`.
#: Values are raw OpenAPI fragments (plain dicts) merged verbatim into the
#: generated operation object.
type OperationMetadata = Mapping[str, Any]
def operation(summary: Optional[str] = None,
description: Optional[str] = None,
tags: Optional[Sequence[str]] = None,
request_body: Optional[Mapping[str, Any]] = None,
responses: Optional[Mapping[int | str, Mapping[str, Any]]] = None,
parameters: Optional[Sequence[Mapping[str, Any]]] = None,
operation_id: Optional[str] = None,
deprecated: bool = False,
hidden: bool = False) -> Callable[[F], F]:
"""Attach OpenAPI metadata to a Kaya route handler.
The metadata is stored on the function itself and picked up by
:class:`~kaya.openapi.OpenAPIMixin` when generating the specification.
All schema fragments are plain dicts inserted verbatim into the generated
OpenAPI document, so any valid OpenAPI 3.1 construct can be used.
Example::
@app.GET('/users/${user_id:int}')
@operation(summary='Get a user',
tags=['users'],
responses={200: {'description': 'The user'},
404: {'description': 'User not found'}})
async def get_user(ctx: HttpContext, user_id: int) -> None:
...
:param summary: short summary of the operation
:param description: longer description (defaults to the handler docstring)
:param tags: list of OpenAPI tags
:param request_body: OpenAPI ``requestBody`` object
:param responses: mapping of status code (or ``'default'``) to OpenAPI
response objects
:param parameters: extra OpenAPI parameter objects merged with the
auto-generated path parameters (entries whose ``name`` matches a path
parameter override the auto-generated one)
:param operation_id: explicit OpenAPI ``operationId``
:param deprecated: mark the operation as deprecated
:param hidden: exclude the operation from the generated specification
"""
def decorator(func: F) -> F:
metadata: dict[str, Any] = {}
if summary is not None:
metadata['summary'] = summary
if description is not None:
metadata['description'] = description
if tags is not None:
metadata['tags'] = list(tags)
if request_body is not None:
metadata['request_body'] = dict(request_body)
if responses is not None:
metadata['responses'] = {str(k): dict(v) for k, v in responses.items()}
if parameters is not None:
metadata['parameters'] = [dict(p) for p in parameters]
if operation_id is not None:
metadata['operation_id'] = operation_id
if deprecated:
metadata['deprecated'] = True
if hidden:
metadata['hidden'] = True
setattr(func, METADATA_ATTR, metadata)
return func
return decorator
def get_metadata(handler: Any) -> OperationMetadata:
"""Return the metadata attached by :func:`operation`, or an empty mapping."""
metadata = getattr(handler, METADATA_ATTR, None)
if isinstance(metadata, Mapping):
return metadata
return {}
@@ -0,0 +1,110 @@
import json
from html import escape
from typing import Any, Mapping, Optional, Sequence
from kaya.core import HttpContext, KayaApp, KayaMixin
from ._spec import generate_spec
_DOCS_PAGE = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>{title} - API documentation</title>
<link rel="stylesheet" href="https://unpkg.com/swagger-ui-dist@5/swagger-ui.css" />
</head>
<body>
<div id="swagger-ui"></div>
<script src="https://unpkg.com/swagger-ui-dist@5/swagger-ui-bundle.js" crossorigin></script>
<script>
window.onload = () => {{
window.ui = SwaggerUIBundle({{
url: '{spec_path}',
dom_id: '#swagger-ui',
}});
}};
</script>
</body>
</html>
"""
class OpenAPIMixin(KayaMixin):
"""Kaya mixin serving an auto-generated OpenAPI specification.
Registers two routes on the app:
- ``spec_path`` (default ``/openapi.json``): the OpenAPI document,
generated on each request from the application's routing tree so that
routes registered after the mixin are always included.
- ``docs_path`` (default ``/docs``): a Swagger UI page rendering the
specification (assets are loaded from a CDN).
Routes are converted as follows:
- ``${name}`` path segments become ``{name}`` string path parameters,
``${name:int}`` become integer path parameters;
- wildcard (``*``) and websocket routes are skipped, since they cannot be
expressed in OpenAPI;
- method-agnostic routes (registered with ``app.route(path)``) are
documented under all standard HTTP methods;
- the mixin's own routes are excluded unless ``include_self`` is true.
Use :func:`~kaya.openapi.operation` to attach summaries, tags, request
bodies and response schemas to individual handlers; the handler docstring
is used as the operation description when no explicit one is given.
Example::
openapi = OpenAPIMixin(title='My API', version='1.0.0')
app = KayaApp(mixins=[openapi])
@app.GET('/users/${user_id:int}')
@operation(summary='Get a user', tags=['users'])
async def get_user(ctx: HttpContext, user_id: int) -> None:
...
"""
def __init__(self,
title: str,
version: str,
description: str = '',
spec_path: str = '/openapi.json',
docs_path: str = '/docs',
servers: Optional[Sequence[Mapping[str, Any]]] = None,
openapi_version: str = '3.1.0',
include_self: bool = False) -> None:
self._title = title
self._version = version
self._description = description
self._spec_path = spec_path
self._docs_path = docs_path
self._servers = servers
self._openapi_version = openapi_version
self._include_self = include_self
def apply(self, app: KayaApp) -> None:
@app.GET(self._spec_path)
async def openapi_spec(ctx: HttpContext) -> None:
spec = self.generate_spec(app)
await ctx.send_str(200,
json.dumps(spec, indent=2),
{'Content-Type': 'application/json'})
@app.GET(self._docs_path)
async def openapi_docs(ctx: HttpContext) -> None:
page = _DOCS_PAGE.format(title=escape(self._title), spec_path=self._spec_path)
await ctx.send_str(200, page, {'Content-Type': 'text/html; charset=utf-8'})
def generate_spec(self, app: KayaApp) -> Mapping[str, Any]:
"""Generate the OpenAPI document for ``app`` (also used by the
``spec_path`` endpoint on every request)."""
exclude_paths = frozenset() if self._include_self else frozenset((self._spec_path, self._docs_path))
return generate_spec(app,
title=self._title,
version=self._version,
description=self._description,
servers=self._servers,
exclude_paths=exclude_paths,
openapi_version=self._openapi_version)
@@ -0,0 +1,177 @@
from copy import deepcopy
from dataclasses import dataclass
from inspect import getdoc
from typing import AbstractSet, Any, Mapping, Optional, Sequence
from kaya.core import HttpMethod, KayaApp
from kaya.core._path_handler import PathHandler
from kaya.core._path_matcher import GlobMatcher, IntMatcher, Node, PathMatcher, StrMatcher
from ._metadata import get_metadata
#: HTTP methods documented for method-agnostic routes
#: (registered with ``app.route(path)`` without explicit methods).
STANDARD_METHODS: Sequence[HttpMethod] = (
HttpMethod.GET,
HttpMethod.PUT,
HttpMethod.POST,
HttpMethod.DELETE,
HttpMethod.OPTIONS,
HttpMethod.HEAD,
HttpMethod.PATCH,
)
@dataclass
class _Route:
raw_path: str
openapi_path: str
method: Optional[HttpMethod]
params: Sequence[Mapping[str, Any]]
handlers: Sequence[PathHandler]
def _walk(node: Node | PathMatcher,
raw_path: str,
openapi_path: str,
params: Sequence[Mapping[str, Any]],
routes: list[_Route]) -> None:
if node.handlers:
routes.append(_Route(raw_path or '/', openapi_path or '/', None, params, list(node.handlers)))
for key, child in node.children.items():
if isinstance(key, HttpMethod):
if key is not HttpMethod.WS and child.handlers:
routes.append(_Route(raw_path or '/', openapi_path or '/', key, params, list(child.handlers)))
else:
_walk(child, f'{raw_path}/{key}', f'{openapi_path}/{key}', params, routes)
for matcher in node.path_matchers:
if isinstance(matcher, GlobMatcher):
# Wildcard routes cannot be expressed in OpenAPI path syntax
continue
if isinstance(matcher, IntMatcher):
raw_segment = f'${{{matcher.name}:int}}'
schema: Mapping[str, Any] = {'type': 'integer'}
elif isinstance(matcher, StrMatcher):
raw_segment = f'${{{matcher.name}}}'
schema = {'type': 'string'}
else:
continue
param: dict[str, Any] = {
'name': matcher.name,
'in': 'path',
'required': True,
'schema': dict(schema),
}
_walk(matcher,
f'{raw_path}/{raw_segment}',
f'{openapi_path}/{{{matcher.name}}}',
(*params, param),
routes)
def _merge_parameters(params: Sequence[Mapping[str, Any]],
extra: Any) -> list[Mapping[str, Any]]:
merged: list[Mapping[str, Any]] = [dict(p) for p in params]
if not isinstance(extra, Sequence) or isinstance(extra, (str, bytes)):
return merged
for candidate in extra:
if not isinstance(candidate, Mapping):
continue
for i, existing in enumerate(merged):
if existing.get('name') == candidate.get('name') and existing.get('in') == candidate.get('in'):
merged[i] = candidate
break
else:
merged.append(candidate)
return merged
def _build_operation(params: Sequence[Mapping[str, Any]],
handler: PathHandler) -> Optional[dict[str, Any]]:
callback = getattr(handler, 'callback', None)
metadata = get_metadata(callback) if callback is not None else {}
if metadata.get('hidden'):
return None
operation: dict[str, Any] = {}
operation_id = metadata.get('operation_id')
if operation_id is not None:
operation['operationId'] = operation_id
summary = metadata.get('summary')
if summary is not None:
operation['summary'] = summary
description = metadata.get('description')
if description is None and callback is not None:
description = getdoc(callback)
if description is not None:
operation['description'] = description
tags = metadata.get('tags')
if tags is not None:
operation['tags'] = tags
if metadata.get('deprecated'):
operation['deprecated'] = True
parameters = _merge_parameters(params, metadata.get('parameters'))
if parameters:
operation['parameters'] = parameters
request_body = metadata.get('request_body')
if request_body is not None:
operation['requestBody'] = request_body
responses = metadata.get('responses')
operation['responses'] = responses if responses is not None else {
'default': {'description': 'Successful response'},
}
return operation
def generate_spec(app: KayaApp,
title: str,
version: str,
description: str = '',
servers: Optional[Sequence[Mapping[str, Any]]] = None,
exclude_paths: AbstractSet[str] = frozenset(),
openapi_version: str = '3.1.0') -> Mapping[str, Any]:
"""Generate an OpenAPI specification document from a :class:`KayaApp`.
The application's routing tree is walked and every route is converted to
an OpenAPI path item:
- static segments and ``${name}`` / ``${name:int}`` parameters are mapped
to OpenAPI path templating (``{name}``);
- wildcard (``*``) and websocket routes are skipped;
- method-agnostic routes are documented under all standard HTTP methods;
- routes whose Kaya path is in ``exclude_paths`` are skipped.
:param app: the application to inspect
:param title: value of ``info.title``
:param version: value of ``info.version``
:param description: value of ``info.description``
:param servers: list of OpenAPI server objects
:param exclude_paths: Kaya paths (e.g. ``/openapi.json``) to omit
:param openapi_version: OpenAPI version to declare
:return: the OpenAPI document as a JSON-serializable mapping
"""
routes: list[_Route] = []
_walk(app._tree.root, '', '', (), routes)
paths: dict[str, dict[str, Any]] = {}
for route in sorted(routes, key=lambda it: it.openapi_path):
if route.raw_path in exclude_paths or not route.handlers:
continue
operation = _build_operation(route.params, route.handlers[0])
if operation is None:
continue
path_item = paths.setdefault(route.openapi_path, {})
methods = STANDARD_METHODS if route.method is None else (route.method,)
for method in methods:
path_item[method.value.lower()] = deepcopy(operation)
info: dict[str, Any] = {'title': title, 'version': version}
if description:
info['description'] = description
spec: dict[str, Any] = {
'openapi': openapi_version,
'info': info,
'paths': paths,
}
if servers:
spec['servers'] = [dict(server) for server in servers]
return spec
+167
View File
@@ -0,0 +1,167 @@
import json
import unittest
from typing import Any, Mapping, Sequence
import httpx
from pwo import async_test
from kaya.core import HttpContext, HttpMethod, KayaApp, WebSocket
from kaya.openapi import OpenAPIMixin, operation
class OpenAPITest(unittest.TestCase):
app: KayaApp
def setUp(self) -> None:
self.app = KayaApp(mixins=[OpenAPIMixin(title='Test API', version='1.2.3')])
@self.app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
"""Say hello."""
await ctx.send_str(200, 'Hello World!')
@self.app.GET('/users/${user_id:int}')
@operation(summary='Get a user',
tags=['users'],
responses={
200: {'description': 'The user'},
404: {'description': 'User not found'},
})
async def get_user(ctx: HttpContext, user_id: int) -> None:
await ctx.send_str(200, str(user_id))
@self.app.POST('/users/${name}')
async def create_user(ctx: HttpContext, name: str) -> None:
await ctx.send_str(201, name)
@self.app.route('/ping')
async def ping(ctx: HttpContext) -> None:
await ctx.send_str(200, 'pong')
@self.app.GET('/files/*', recursive=True)
async def serve_file(ctx: HttpContext, path: Sequence[str]) -> None:
await ctx.send_str(200, '/'.join(path))
@self.app.GET('/internal/health')
@operation(hidden=True)
async def health(ctx: HttpContext) -> None:
await ctx.send_str(200, 'ok')
@self.app.websocket('/echo')
async def echo(ws: WebSocket) -> None:
await ws.accept()
await ws.close()
async def _get_spec(self) -> Mapping[str, Any]:
transport = httpx.ASGITransport(app=self.app)
async with httpx.AsyncClient(transport=transport, base_url='http://127.0.0.1:80') as client:
response = await client.get('/openapi.json')
self.assertEqual(200, response.status_code)
self.assertEqual('application/json', response.headers['Content-Type'])
return json.loads(response.text)
@async_test
async def test_spec_endpoint(self) -> None:
spec = await self._get_spec()
self.assertEqual('3.1.0', spec['openapi'])
self.assertEqual({'title': 'Test API', 'version': '1.2.3'}, spec['info'])
self.assertIn('paths', spec)
@async_test
async def test_static_route_with_docstring(self) -> None:
spec = await self._get_spec()
hello = spec['paths']['/hello']['get']
self.assertEqual('Say hello.', hello['description'])
self.assertIn('responses', hello)
@async_test
async def test_int_path_parameter(self) -> None:
spec = await self._get_spec()
operation = spec['paths']['/users/{user_id}']['get']
self.assertEqual('Get a user', operation['summary'])
self.assertEqual(['users'], operation['tags'])
self.assertEqual({
'200': {'description': 'The user'},
'404': {'description': 'User not found'},
}, operation['responses'])
self.assertEqual(
[{'name': 'user_id', 'in': 'path', 'required': True, 'schema': {'type': 'integer'}}],
operation['parameters'])
@async_test
async def test_str_path_parameter(self) -> None:
spec = await self._get_spec()
operation = spec['paths']['/users/{name}']['post']
self.assertEqual(
[{'name': 'name', 'in': 'path', 'required': True, 'schema': {'type': 'string'}}],
operation['parameters'])
@async_test
async def test_method_agnostic_route(self) -> None:
spec = await self._get_spec()
path_item = spec['paths']['/ping']
for method in ('get', 'put', 'post', 'delete', 'options', 'head', 'patch'):
self.assertIn(method, path_item)
@async_test
async def test_excluded_routes(self) -> None:
spec = await self._get_spec()
paths = spec['paths']
# wildcard routes cannot be expressed in OpenAPI
self.assertNotIn('/files/*', paths)
self.assertFalse(any('files' in path for path in paths))
# websocket routes are not part of OpenAPI
self.assertNotIn('/echo', paths)
# hidden operations are skipped
self.assertNotIn('/internal/health', paths)
# the mixin's own endpoints are excluded by default
self.assertNotIn('/openapi.json', paths)
self.assertNotIn('/docs', paths)
@async_test
async def test_docs_endpoint(self) -> None:
transport = httpx.ASGITransport(app=self.app)
async with httpx.AsyncClient(transport=transport, base_url='http://127.0.0.1:80') as client:
response = await client.get('/docs')
self.assertEqual(200, response.status_code)
self.assertEqual('text/html; charset=utf-8', response.headers['Content-Type'])
self.assertIn('swagger-ui', response.text)
self.assertIn('/openapi.json', response.text)
@async_test
async def test_late_registered_routes_are_included(self) -> None:
@self.app.GET('/late')
async def late(ctx: HttpContext) -> None:
await ctx.send_str(200, 'late')
spec = await self._get_spec()
self.assertIn('/late', spec['paths'])
@async_test
async def test_custom_paths_and_self_inclusion(self) -> None:
app = KayaApp(mixins=[OpenAPIMixin(title='Custom',
version='0.1.0',
spec_path='/spec.json',
docs_path='/swagger',
include_self=True)])
@app.route('/items/${item_id:int}', HttpMethod.DELETE)
async def delete_item(ctx: HttpContext, item_id: int) -> None:
await ctx.send_empty(204)
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url='http://127.0.0.1:80') as client:
response = await client.get('/spec.json')
self.assertEqual(200, response.status_code)
spec = json.loads(response.text)
self.assertIn('delete', spec['paths']['/items/{item_id}'])
self.assertIn('/spec.json', spec['paths'])
self.assertIn('/swagger', spec['paths'])
response = await client.get('/swagger')
self.assertEqual(200, response.status_code)
self.assertIn('/spec.json', response.text)
if __name__ == '__main__':
unittest.main()
+85
View File
@@ -0,0 +1,85 @@
# kaya-otel
OpenTelemetry tracing and metrics for the Kaya web framework.
Provides `OTelMixin`, a `KayaMixin` that instruments HTTP requests and
WebSocket connections with OpenTelemetry spans and exports HTTP server
metrics, shipping everything to an OTLP/HTTP collector.
## Usage
```python
from kaya.core import KayaApp, HttpContext
from kaya.otel import OTelMixin
app = KayaApp(mixins=[
OTelMixin(
service_name='my-service',
endpoint='http://localhost:4318',
)
])
@app.GET('/')
async def home(ctx: HttpContext):
await ctx.send_str(200, 'Hello World!')
```
## Parameters
- `service_name`: value of the `service.name` resource attribute.
- `endpoint`: base URL of an OTLP/HTTP collector; the signal paths
`/v1/traces` and `/v1/metrics` are appended. When `None`, the exporters
use their own defaults, including the standard
`OTEL_EXPORTER_OTLP_ENDPOINT` environment variable.
- `headers`: extra HTTP headers sent to the collector (e.g. authentication).
- `metric_export_interval_millis`: metric export interval
(default `60000`).
- `tracer_provider` / `meter_provider`: inject custom providers (e.g. with
in-memory exporters for tests) instead of the OTLP defaults. The mixin
only shuts down providers it created itself.
- `resource_attributes`: extra resource attributes merged with
`service.name` when the mixin creates the providers.
- `excluded_paths` / `excluded_path_regexes`: skip tracing and metrics for
matching paths (useful for health checks and metrics endpoints).
- `capture_request_headers` / `capture_response_headers`: opt-in HTTP header
capture as `http.request.header.<name>` / `http.response.header.<name>`
span attributes. Header names are normalized to lowercase with `-`
replaced by `_`; values are captured as string lists.
- `sanitize_headers`: headers captured as `REDACTED` (default:
`authorization`, `proxy-authorization`, `cookie`, `set-cookie`).
- `server_request_hook` / `server_response_hook` / `websocket_connect_hook` /
`websocket_close_hook`: optional synchronous callbacks invoked with the
span and the Kaya context/websocket at the corresponding lifecycle point.
Hook exceptions are logged and do not fail the request.
- `metrics_include_raw_path`: when `True`, metrics use raw `url.path`
attributes (legacy, potentially high cardinality). The default `False`
keeps metrics low-cardinality by using `http.route` when Kaya can resolve
a route template.
- `websocket_error_close_codes`: close codes that mark a websocket span as
failed. Defaults to protocol/application error codes such as `1002`,
`1003` and `1007`-`1011`.
## Behavior
- Every HTTP request gets a `SERVER` span named `<METHOD> <path>` with the
usual HTTP semantic attributes (`http.request.method`, `url.path`,
`client.address`, `server.address`, ...). When Kaya resolves a route
template, the span name is updated to `<METHOD> <route template>` and
`http.route` is set. The response status code is recorded as
`http.response.status_code` when the handler sends the response; 5xx
statuses mark the span as failed. Exceptions escaping the handler are
recorded on the span and also mark it as failed.
- Every WebSocket connection gets one span for its whole lifetime. The close
code is recorded as `kaya.websocket.close_code`; exceptions and configured
error close codes mark the span as failed.
- W3C `traceparent`/`tracestate` headers on incoming requests are honored,
so traces propagate from upstream services.
- Metrics:
- `http.server.request.duration` (histogram, seconds), with
`http.request.method`, `http.route` when known and
`http.response.status_code` attributes.
- `http.server.active_requests` (up-down counter), with
`http.request.method` attributes.
`OTelMixin` is a `KayaMixin`, so the app stays a `KayaApp` and both ASGI and
RSGI keep working.
+58
View File
@@ -0,0 +1,58 @@
[build-system]
requires = ["setuptools>=61.0", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"
[project]
name = "kaya-otel"
dynamic = ["version"]
authors = [
{ name="Walter Oggioni", email="oggioni.walter@gmail.com" },
]
description = "OpenTelemetry tracing and metrics for the Kaya lightweight ASGI web framework"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
classifiers = [
'Development Status :: 3 - Alpha',
'Topic :: Utilities',
'Intended Audience :: System Administrators',
'Intended Audience :: Developers',
'Environment :: Console',
'Programming Language :: Python :: 3',
]
dependencies = [
"kaya-core",
"opentelemetry-sdk",
"opentelemetry-exporter-otlp-proto-http",
]
[project.optional-dependencies]
dev = [
"build", "mypy", "ipdb", "twine", "httpx", "httpx-ws", "kaya-rsgi"
]
[project.urls]
"Homepage" = "https://github.com/woggioni/kaya"
"Bug Tracker" = "https://github.com/woggioni/kaya/issues"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
[tool.mypy]
python_version = "3.12"
disallow_untyped_defs = true
show_error_codes = true
no_implicit_optional = true
warn_return_any = true
warn_unused_ignores = true
exclude = ["scripts", "docs", "test"]
strict = true
[tool.setuptools_scm]
root = "../.."
version_file = "src/kaya/otel/_version.py"
[tool.setuptools_scm.tag]
prefix = "release/"
@@ -0,0 +1,10 @@
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
from ._mixin import OTelMixin
__all__ = [
'OTelMixin',
]
+491
View File
@@ -0,0 +1,491 @@
import re
import time
from asyncio import AbstractEventLoop
from logging import getLogger
from pathlib import Path
from typing import (
Any,
AsyncGenerator,
Callable,
Dict,
Mapping,
Optional,
Sequence,
Tuple,
)
from kaya.core import HttpContext, KayaApp, KayaMixin, WebSocket, WebSocketMessage
from kaya.core._types import StrOrStrings
from opentelemetry import context as otel_context
from opentelemetry import trace
from opentelemetry.context import Context
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.metrics import Histogram, UpDownCounter
from opentelemetry.propagators.textmap import Getter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.trace import Span, SpanKind, Status, StatusCode, Tracer
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
log = getLogger(__name__)
RequestHook = Callable[[Span, HttpContext], None]
WebSocketHook = Callable[[Span, WebSocket], None]
_DEFAULT_SANITIZE_HEADERS = (
'authorization',
'proxy-authorization',
'cookie',
'set-cookie',
)
_DEFAULT_WEBSOCKET_ERROR_CLOSE_CODES = frozenset((1002, 1003, 1007, 1008, 1009, 1010, 1011))
class _HeadersGetter(Getter[Mapping[str, Sequence[str]]]):
"""Extract propagation headers from a kaya request/websocket context.
Kaya header mappings have lowercase names with one or more values each,
which is all the W3C trace-context propagator needs.
"""
def get(self, carrier: Mapping[str, Sequence[str]], key: str) -> Optional[list[str]]:
values = carrier.get(key.lower())
if not values:
return None
return list(values)
def keys(self, carrier: Mapping[str, Sequence[str]]) -> list[str]:
return list(carrier.keys())
_PROPAGATOR = TraceContextTextMapPropagator()
_GETTER = _HeadersGetter()
def _server_attributes(ctx: HttpContext) -> Dict[str, Any]:
attributes: Dict[str, Any] = {
'http.request.method': str(ctx.method),
'url.scheme': ctx.scheme,
'url.path': ctx.path,
}
if ctx.query_string:
attributes['url.query'] = ctx.query_string
if ctx.client is not None:
attributes['client.address'] = ctx.client[0]
attributes['client.port'] = ctx.client[1]
if ctx.server is not None:
attributes['server.address'] = ctx.server[0]
if ctx.server[1] is not None:
attributes['server.port'] = ctx.server[1]
return attributes
def _normalize_header_name(name: str) -> str:
return name.lower().replace('-', '_')
def _header_values(value: StrOrStrings | Sequence[str]) -> list[str]:
if isinstance(value, str):
return [value]
return list(value)
class _SpanHolder:
"""Mutable per-request slot shared by the tracing hooks and wrappers."""
__slots__ = ('span', 'token', 'status_code', 'start_ns', 'active_attributes', 'close_code')
def __init__(self, span: Span, token: object, active_attributes: Dict[str, Any]) -> None:
self.span = span
self.token = token
self.status_code: Optional[int] = None
self.start_ns = time.perf_counter_ns()
self.active_attributes = active_attributes
self.close_code: Optional[int] = None
class _TracedHttpContext(HttpContext):
"""HttpContext wrapper that records the response status code on the span.
Attributes not explicitly overridden are delegated to the wrapped context
via ``__getattr__``, so it works with any concrete ``HttpContext`` (ASGI
or RSGI) and preserves attributes set by other mixins (e.g. ``session``).
"""
def __init__(self, ctx: HttpContext, holder: _SpanHolder, mixin: 'OTelMixin') -> None:
object.__setattr__(self, '_ctx', ctx)
object.__setattr__(self, 'session', ctx.session)
object.__setattr__(self, '_holder', holder)
object.__setattr__(self, '_mixin', mixin)
def __getattr__(self, name: str) -> Any:
if name == '_ctx':
raise AttributeError(name)
return getattr(self._ctx, name)
def _sent(self, status: int, headers: Optional[Mapping[str, StrOrStrings]]) -> None:
self._holder.status_code = status
self._holder.span.set_attribute('http.response.status_code', status)
self._mixin._capture_response_headers(self._holder.span, headers)
async def stream_body(self,
status: int,
body_generator: AsyncGenerator[bytes, None],
headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.stream_body(status, body_generator, headers)
async def send_bytes(self, status: int, body: bytes, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_bytes(status, body, headers)
async def send_str(self, status: int, body: str, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_str(status, body, headers)
async def send_file(self, status: int, path: Path, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_file(status, path, headers)
async def send_empty(self, status: int, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
self._sent(status, headers)
await self._ctx.send_empty(status, headers)
class _TracedWebSocket(WebSocket):
"""WebSocket wrapper that keeps the connection span reachable.
Everything is delegated to the wrapped socket via ``__getattr__``, so it
works with any concrete ``WebSocket`` (ASGI or RSGI) and preserves
attributes set by other mixins (e.g. ``session``).
"""
def __init__(self, ws: WebSocket, holder: _SpanHolder) -> None:
object.__setattr__(self, '_ws', ws)
object.__setattr__(self, 'session', ws.session)
object.__setattr__(self, '_holder', holder)
def __getattr__(self, name: str) -> Any:
if name == '_ws':
raise AttributeError(name)
return getattr(self._ws, name)
async def accept(self, headers: Optional[Mapping[str, StrOrStrings]] = None) -> None:
await self._ws.accept(headers)
async def receive(self) -> WebSocketMessage:
return await self._ws.receive()
async def send_text(self, data: str) -> None:
await self._ws.send_text(data)
async def send_bytes(self, data: bytes) -> None:
await self._ws.send_bytes(data)
async def close(self, code: int = 1000) -> None:
self._holder.close_code = code
await self._ws.close(code)
async def __anext__(self) -> WebSocketMessage:
return await self._ws.__anext__()
class OTelMixin(KayaMixin):
"""Kaya mixin adding OpenTelemetry tracing and metrics.
HTTP requests get a ``SERVER`` span named ``<METHOD> <path>`` carrying the
usual HTTP semantic attributes; the response status code is recorded when
the handler sends the response and 5xx statuses mark the span as failed.
Exceptions escaping the handler are recorded and mark the span as failed.
WebSocket connections get one span for the whole connection lifetime.
W3C ``traceparent``/``tracestate`` headers on incoming requests are
honored, so traces propagate from upstream services.
Metrics exported on the meter provider:
- ``http.server.request.duration`` (histogram, seconds), with
``http.request.method``, ``http.route`` when known and
``http.response.status_code`` attributes;
- ``http.server.active_requests`` (up-down counter), with
``http.request.method`` attributes.
Example::
app = KayaApp(mixins=[
OTelMixin(
service_name='my-service',
endpoint='http://localhost:4318',
)
])
``endpoint`` is the base URL of an OTLP/HTTP collector (the signal paths
``/v1/traces`` and ``/v1/metrics`` are appended); when ``None`` the
exporters fall back to their own defaults, including the standard
``OTEL_EXPORTER_OTLP_ENDPOINT`` environment variable. Custom
``tracer_provider``/``meter_provider`` instances (e.g. with in-memory
exporters for tests) can be injected instead of the OTLP defaults; the
mixin only shuts down providers it created itself.
"""
def __init__(self,
service_name: str,
endpoint: Optional[str] = None,
headers: Optional[Mapping[str, str]] = None,
metric_export_interval_millis: float = 60000,
tracer_provider: Optional[TracerProvider] = None,
meter_provider: Optional[MeterProvider] = None,
resource_attributes: Optional[Mapping[str, Any]] = None,
excluded_paths: Sequence[str] = (),
excluded_path_regexes: Sequence[str] = (),
capture_request_headers: Sequence[str] = (),
capture_response_headers: Sequence[str] = (),
sanitize_headers: Sequence[str] = _DEFAULT_SANITIZE_HEADERS,
server_request_hook: Optional[RequestHook] = None,
server_response_hook: Optional[RequestHook] = None,
websocket_connect_hook: Optional[WebSocketHook] = None,
websocket_close_hook: Optional[WebSocketHook] = None,
metrics_include_raw_path: bool = False,
websocket_error_close_codes: Optional[Sequence[int]] = None) -> None:
self._owns_tracer_provider = tracer_provider is None
self._owns_meter_provider = meter_provider is None
if tracer_provider is None:
tracer_provider = TracerProvider(resource=self._create_resource(service_name, resource_attributes))
span_exporter = OTLPSpanExporter(
endpoint=f'{endpoint}/v1/traces' if endpoint else None,
headers=dict(headers) if headers else None,
)
tracer_provider.add_span_processor(BatchSpanProcessor(span_exporter))
if meter_provider is None:
meter_provider = MeterProvider(
resource=self._create_resource(service_name, resource_attributes),
metric_readers=[
PeriodicExportingMetricReader(
OTLPMetricExporter(
endpoint=f'{endpoint}/v1/metrics' if endpoint else None,
headers=dict(headers) if headers else None,
),
export_interval_millis=metric_export_interval_millis,
)
],
)
self._tracer_provider = tracer_provider
self._meter_provider = meter_provider
self._tracer: Tracer = tracer_provider.get_tracer('kaya.otel')
meter = meter_provider.get_meter('kaya.otel')
self._request_duration: Histogram = meter.create_histogram(
'http.server.request.duration',
unit='s',
description='Duration of HTTP server requests',
)
self._active_requests: UpDownCounter = meter.create_up_down_counter(
'http.server.active_requests',
unit='{request}',
description='Number of in-flight HTTP server requests',
)
self._app: Optional[KayaApp] = None
self._excluded_paths = frozenset(excluded_paths)
self._excluded_path_regexes = tuple(re.compile(regex) for regex in excluded_path_regexes)
self._request_headers_to_capture = tuple(capture_request_headers)
self._response_headers_to_capture = tuple(capture_response_headers)
self._sanitize_headers = frozenset(name.lower() for name in sanitize_headers)
self._server_request_hook = server_request_hook
self._server_response_hook = server_response_hook
self._websocket_connect_hook = websocket_connect_hook
self._websocket_close_hook = websocket_close_hook
self._metrics_include_raw_path = metrics_include_raw_path
self._websocket_error_close_codes = (
frozenset(websocket_error_close_codes)
if websocket_error_close_codes is not None
else _DEFAULT_WEBSOCKET_ERROR_CLOSE_CODES
)
# Live spans keyed by the id of the (wrapped) context/websocket the
# after hooks receive; entries are removed when the span ends.
self._spans: Dict[int, _SpanHolder] = {}
@staticmethod
def _create_resource(service_name: str,
resource_attributes: Optional[Mapping[str, Any]]) -> Resource:
attributes: Dict[str, Any] = {'service.name': service_name}
if resource_attributes:
attributes.update(resource_attributes)
return Resource.create(attributes)
def apply(self, app: KayaApp) -> None:
self._app = app
app.add_before_request_hook(self._before_request)
app.add_after_request_hook(self._after_request)
app.add_before_websocket_hook(self._before_websocket)
app.add_after_websocket_hook(self._after_websocket)
def _is_excluded(self, path: str) -> bool:
if path in self._excluded_paths:
return True
return any(regex.search(path) is not None for regex in self._excluded_path_regexes)
@staticmethod
def _call_hook(hook: Optional[Callable[[Span, Any], None]], span: Span, value: Any) -> None:
if hook is None:
return
try:
hook(span, value)
except Exception:
log.exception('OpenTelemetry span hook failed')
def _capture_headers(self,
span: Span,
headers: Mapping[str, StrOrStrings | Sequence[str]],
configured: Sequence[str],
prefix: str) -> None:
if not configured:
return
lower_headers = {name.lower(): value for name, value in headers.items()}
for name in configured:
key = name.lower()
if key not in lower_headers:
continue
values = _header_values(lower_headers[key])
if key in self._sanitize_headers:
values = ['REDACTED'] * len(values)
span.set_attribute(f'{prefix}.{_normalize_header_name(name)}', values)
def _capture_request_headers(self, span: Span, headers: Mapping[str, Sequence[str]]) -> None:
self._capture_headers(span, headers, self._request_headers_to_capture, 'http.request.header')
def _capture_response_headers(self,
span: Span,
headers: Optional[Mapping[str, StrOrStrings]]) -> None:
if headers is None:
return
self._capture_headers(span, headers, self._response_headers_to_capture, 'http.response.header')
def _start_span(self, name: str, attributes: Mapping[str, Any],
headers: Mapping[str, Sequence[str]]) -> _SpanHolder:
parent: Context = _PROPAGATOR.extract(carrier=headers, getter=_GETTER)
span = self._tracer.start_span(
name,
context=parent,
kind=SpanKind.SERVER,
attributes=dict(attributes),
)
self._capture_request_headers(span, headers)
token = otel_context.attach(trace.set_span_in_context(span, parent))
return _SpanHolder(span, token, {})
@staticmethod
def _finish_span(holder: _SpanHolder, error: bool) -> None:
if error:
holder.span.set_status(Status(StatusCode.ERROR))
holder.span.end()
otel_context.detach(holder.token) # type: ignore[arg-type]
def _pop_holder(self, obj: Any) -> Optional[_SpanHolder]:
"""Find and remove the span holder for a context/websocket.
The before hooks key holders by the id of the context they received,
but after hooks run on the outermost wrapper (later mixins may have
wrapped the context again), so the id may not match. In that case the
holder is reachable through the wrapper delegation chain as
``_holder`` and the stale dict entry is reaped by identity.
"""
holder = self._spans.pop(id(obj), None)
if holder is not None:
return holder
candidate = getattr(obj, '_holder', None)
if not isinstance(candidate, _SpanHolder):
return None
for key, value in list(self._spans.items()):
if value is candidate:
del self._spans[key]
return candidate
def _route_template(self, ctx: HttpContext) -> Optional[str]:
if self._app is None:
return None
return self._app.route_template(ctx.path, ctx.method)
async def _before_request(self, ctx: HttpContext) -> Optional[HttpContext]:
if self._is_excluded(ctx.path):
return None
holder = self._start_span(
f'{ctx.method} {ctx.path}',
_server_attributes(ctx),
ctx.headers,
)
holder.active_attributes = {'http.request.method': str(ctx.method)}
if self._metrics_include_raw_path:
holder.active_attributes['url.path'] = ctx.path
self._spans[id(ctx)] = holder
self._active_requests.add(1, holder.active_attributes)
self._call_hook(self._server_request_hook, holder.span, ctx)
return _TracedHttpContext(ctx, holder, self)
async def _after_request(self, ctx: HttpContext) -> None:
holder = self._pop_holder(ctx)
if holder is None:
return
status_code = holder.status_code
exception = getattr(ctx, 'exception', None)
error = status_code is not None and status_code >= 500
if isinstance(exception, BaseException):
holder.span.record_exception(exception)
error = True
route_template = self._route_template(ctx)
if route_template is not None:
holder.span.update_name(f'{ctx.method} {route_template}')
holder.span.set_attribute('http.route', route_template)
self._call_hook(self._server_response_hook, holder.span, ctx)
self._finish_span(holder, error)
self._active_requests.add(-1, holder.active_attributes)
metric_attributes: Dict[str, Any] = {
'http.request.method': str(ctx.method),
}
if self._metrics_include_raw_path:
metric_attributes['url.path'] = ctx.path
elif route_template is not None:
metric_attributes['http.route'] = route_template
if status_code is not None:
metric_attributes['http.response.status_code'] = status_code
duration = (time.perf_counter_ns() - holder.start_ns) / 1e9
self._request_duration.record(duration, metric_attributes)
async def _before_websocket(self, ws: WebSocket) -> Optional[WebSocket]:
if self._is_excluded(ws.path):
return None
holder = self._start_span(
f'WS {ws.path}',
{
'url.scheme': ws.scheme,
'url.path': ws.path,
},
ws.headers,
)
self._spans[id(ws)] = holder
self._call_hook(self._websocket_connect_hook, holder.span, ws)
return _TracedWebSocket(ws, holder)
async def _after_websocket(self, ws: WebSocket) -> None:
holder = self._pop_holder(ws)
if holder is None:
return
exception = getattr(ws, 'exception', None)
error = isinstance(exception, BaseException)
if isinstance(exception, BaseException):
holder.span.record_exception(exception)
close_code = holder.close_code
if close_code is not None:
holder.span.set_attribute('kaya.websocket.close_code', close_code)
error = error or close_code in self._websocket_error_close_codes
self._call_hook(self._websocket_close_hook, holder.span, ws)
self._finish_span(holder, error)
def shutdown(self, loop: AbstractEventLoop) -> None:
# Only providers created by this mixin are shut down; injected ones
# stay under the caller's control.
if self._owns_tracer_provider:
self._tracer_provider.shutdown()
if self._owns_meter_provider:
self._meter_provider.shutdown()
+394
View File
@@ -0,0 +1,394 @@
import json
import unittest
from typing import Any, Optional, Tuple
import httpx
from pwo import async_test
from kaya.core import HttpContext, KayaApp
from kaya.core._asgi import AsgiWebSocket
from kaya.otel import OTelMixin
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import InMemoryMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import StatusCode
TRACEPARENT = '00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01'
def make_mixin(**kwargs) -> Tuple[OTelMixin, InMemorySpanExporter, InMemoryMetricReader]:
resource = Resource.create({'service.name': 'test-service'})
span_exporter = InMemorySpanExporter()
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter))
metric_reader = InMemoryMetricReader()
meter_provider = MeterProvider(resource=resource, metric_readers=[metric_reader])
mixin = OTelMixin(
service_name='test-service',
tracer_provider=tracer_provider,
meter_provider=meter_provider,
**kwargs,
)
return mixin, span_exporter, metric_reader
def make_app(mixin: OTelMixin) -> KayaApp:
app = KayaApp(mixins=[mixin])
@app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
await ctx.send_str(200, json.dumps({'ok': True}))
@app.GET('/boom')
async def boom(ctx: HttpContext) -> None:
await ctx.send_str(500, 'boom')
return app
async def request(app: KayaApp, path: str = '/hello', headers: Optional[dict[str, str]] = None) -> httpx.Response:
transport = httpx.ASGITransport(app=app, client=('127.0.0.1', 123))
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as http_client:
return await http_client.get(path, headers=headers)
class HttpTracingTest(unittest.TestCase):
@async_test
async def test_request_produces_server_span(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
r = await request(app)
self.assertEqual(200, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual('GET /hello', span.name)
assert span.attributes is not None
self.assertEqual('GET', span.attributes['http.request.method'])
self.assertEqual('/hello', span.attributes['url.path'])
self.assertEqual(200, span.attributes['http.response.status_code'])
self.assertEqual(StatusCode.UNSET, span.status.status_code)
@async_test
async def test_5xx_marks_span_as_error(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
r = await request(app, '/boom')
self.assertEqual(500, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(500, spans[0].attributes['http.response.status_code']) # type: ignore[index]
self.assertEqual(StatusCode.ERROR, spans[0].status.status_code)
@async_test
async def test_traceparent_header_propagates(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
await request(app, headers={'traceparent': TRACEPARENT})
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual(0x4bf92f3577b34da6a3ce929d0e0e4736, span.context.trace_id) # type: ignore[union-attr]
assert span.parent is not None
self.assertEqual(0x00f067aa0ba902b7, span.parent.span_id)
@async_test
async def test_unmatched_route_still_traced(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
r = await request(app, '/nowhere')
self.assertEqual(404, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(404, spans[0].attributes['http.response.status_code']) # type: ignore[index]
@async_test
async def test_no_leaked_spans_after_requests(self):
mixin, span_exporter, _ = make_mixin()
app = make_app(mixin)
await request(app)
await request(app, '/boom')
self.assertEqual(0, len(mixin._spans))
self.assertEqual(2, len(span_exporter.get_finished_spans()))
@async_test
async def test_exception_marks_span_as_error(self):
mixin, span_exporter, _ = make_mixin()
app = KayaApp(mixins=[mixin])
@app.GET('/raises')
async def raises(ctx: HttpContext) -> None:
raise RuntimeError('boom')
transport = httpx.ASGITransport(app=app, client=('127.0.0.1', 123))
async with httpx.AsyncClient(transport=transport, base_url="http://127.0.0.1:80") as http_client:
with self.assertRaises(RuntimeError):
await http_client.get('/raises')
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual(StatusCode.ERROR, span.status.status_code)
self.assertTrue(any(event.name == 'exception' for event in span.events))
self.assertEqual(0, len(mixin._spans))
@async_test
async def test_excluded_path_skips_tracing_and_metrics(self):
mixin, span_exporter, metric_reader = make_mixin(excluded_paths=('/health',))
app = KayaApp(mixins=[mixin])
@app.GET('/health')
async def health(ctx: HttpContext) -> None:
await ctx.send_str(200, 'ok')
@app.GET('/hello')
async def hello(ctx: HttpContext) -> None:
await ctx.send_str(200, 'hello')
await request(app, '/health')
await request(app, '/hello')
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual('GET /hello', spans[0].name)
metrics = MetricsTest._metric_names(metric_reader)
duration_points = list(metrics['http.server.request.duration'].data.data_points)
self.assertEqual(1, len(duration_points))
self.assertEqual(1, duration_points[0].count)
@async_test
async def test_route_template_used_for_span_name_and_metrics(self):
mixin, span_exporter, metric_reader = make_mixin()
app = KayaApp(mixins=[mixin])
@app.GET('/items/${item_id:int}')
async def item(ctx: HttpContext, item_id: int) -> None:
await ctx.send_str(200, str(item_id))
r = await request(app, '/items/123')
self.assertEqual(200, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
span = spans[0]
self.assertEqual('GET /items/${item_id:int}', span.name)
assert span.attributes is not None
self.assertEqual('/items/${item_id:int}', span.attributes['http.route'])
self.assertEqual('/items/123', span.attributes['url.path'])
metrics = MetricsTest._metric_names(metric_reader)
duration_points = list(metrics['http.server.request.duration'].data.data_points)
self.assertEqual(1, len(duration_points))
duration_attributes = dict(duration_points[0].attributes)
self.assertEqual('GET', duration_attributes['http.request.method'])
self.assertEqual('/items/${item_id:int}', duration_attributes['http.route'])
self.assertEqual(200, duration_attributes['http.response.status_code'])
self.assertNotIn('url.path', duration_attributes)
active_points = list(metrics['http.server.active_requests'].data.data_points)
self.assertEqual(1, len(active_points))
self.assertEqual({'http.request.method': 'GET'}, dict(active_points[0].attributes))
@async_test
async def test_header_capture_and_sanitization(self):
mixin, span_exporter, _ = make_mixin(
capture_request_headers=('X-Request-Id', 'Authorization'),
capture_response_headers=('X-Response-Id', 'Set-Cookie'),
)
app = KayaApp(mixins=[mixin])
@app.GET('/headers')
async def headers(ctx: HttpContext) -> None:
await ctx.send_str(200, 'ok', {
'X-Response-Id': 'res-1',
'Set-Cookie': 'sid=secret',
})
r = await request(app, '/headers', headers={
'X-Request-Id': 'req-1',
'Authorization': 'Bearer secret',
})
self.assertEqual(200, r.status_code)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
attributes = spans[0].attributes
assert attributes is not None
self.assertEqual(['req-1'], list(attributes['http.request.header.x_request_id']))
self.assertEqual(['REDACTED'], list(attributes['http.request.header.authorization']))
self.assertEqual(['res-1'], list(attributes['http.response.header.x_response_id']))
self.assertEqual(['REDACTED'], list(attributes['http.response.header.set_cookie']))
@async_test
async def test_http_hooks_are_called(self):
calls = []
def request_hook(span: Any, ctx: HttpContext) -> None:
calls.append(('request', ctx.path))
span.set_attribute('test.request_hook', True)
def response_hook(span: Any, ctx: HttpContext) -> None:
calls.append(('response', ctx.path))
span.set_attribute('test.response_hook', True)
mixin, span_exporter, _ = make_mixin(
server_request_hook=request_hook,
server_response_hook=response_hook,
)
app = make_app(mixin)
await request(app)
self.assertEqual([('request', '/hello'), ('response', '/hello')], calls)
attributes = span_exporter.get_finished_spans()[0].attributes
assert attributes is not None
self.assertTrue(attributes['test.request_hook'])
self.assertTrue(attributes['test.response_hook'])
class MetricsTest(unittest.TestCase):
@staticmethod
def _metric_names(metric_reader: InMemoryMetricReader) -> dict:
data = metric_reader.get_metrics_data()
return {m.name: m for rm in data.resource_metrics for m in rm.scope_metrics[0].metrics
if rm.scope_metrics}
@async_test
async def test_duration_histogram_and_active_requests(self):
mixin, _, metric_reader = make_mixin()
app = make_app(mixin)
await request(app)
metrics = self._metric_names(metric_reader)
self.assertIn('http.server.request.duration', metrics)
self.assertIn('http.server.active_requests', metrics)
duration_points = list(metrics['http.server.request.duration'].data.data_points)
self.assertEqual(1, len(duration_points))
self.assertEqual(1, duration_points[0].count)
self.assertGreaterEqual(duration_points[0].sum, 0)
active_points = list(metrics['http.server.active_requests'].data.data_points)
self.assertEqual(1, len(active_points))
self.assertEqual(0, active_points[0].value)
class WebSocketTracingTest(unittest.TestCase):
@staticmethod
def _make_ws(headers=()):
async def send(message):
pass
async def receive():
return {'type': 'websocket.connect'}
scope = {
'type': 'websocket',
'path': '/ws/games/abc',
'query_string': b'',
'scheme': 'ws',
'client': ('127.0.0.1', 12345),
'server': ('127.0.0.1', 80),
'headers': headers,
}
return AsgiWebSocket(scope, receive, send)
@async_test
async def test_websocket_lifecycle_produces_span(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
self.assertIsNotNone(wrapped)
self.assertEqual(1, len(mixin._spans))
await mixin._after_websocket(wrapped) # type: ignore[arg-type]
self.assertEqual(0, len(mixin._spans))
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual('WS /ws/games/abc', spans[0].name)
@async_test
async def test_websocket_traceparent_propagates(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws(headers=[(b'traceparent', TRACEPARENT.encode())])
wrapped = await mixin._before_websocket(ws)
await mixin._after_websocket(wrapped) # type: ignore[arg-type]
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(0x4bf92f3577b34da6a3ce929d0e0e4736, spans[0].context.trace_id) # type: ignore[union-attr]
@async_test
async def test_wrapped_websocket_delegates(self):
mixin, _, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
self.assertEqual('/ws/games/abc', wrapped.path)
self.assertEqual(('127.0.0.1', 12345), wrapped.client)
await mixin._after_websocket(wrapped)
@async_test
async def test_websocket_error_close_code_marks_span_error(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
await wrapped.close(1011)
await mixin._after_websocket(wrapped)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(StatusCode.ERROR, spans[0].status.status_code)
assert spans[0].attributes is not None
self.assertEqual(1011, spans[0].attributes['kaya.websocket.close_code'])
@async_test
async def test_websocket_exception_marks_span_error(self):
mixin, span_exporter, _ = make_mixin()
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
wrapped.exception = RuntimeError('ws boom')
await mixin._after_websocket(wrapped)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(StatusCode.ERROR, spans[0].status.status_code)
self.assertTrue(any(event.name == 'exception' for event in spans[0].events))
@async_test
async def test_websocket_hooks_are_called(self):
calls = []
def connect_hook(span: Any, ws: Any) -> None:
calls.append(('connect', ws.path))
span.set_attribute('test.websocket_connect_hook', True)
def close_hook(span: Any, ws: Any) -> None:
calls.append(('close', ws.path))
span.set_attribute('test.websocket_close_hook', True)
mixin, span_exporter, _ = make_mixin(
websocket_connect_hook=connect_hook,
websocket_close_hook=close_hook,
)
ws = self._make_ws()
wrapped = await mixin._before_websocket(ws)
assert wrapped is not None
await wrapped.close(1000)
await mixin._after_websocket(wrapped)
self.assertEqual([('connect', '/ws/games/abc'), ('close', '/ws/games/abc')], calls)
spans = span_exporter.get_finished_spans()
self.assertEqual(1, len(spans))
self.assertEqual(StatusCode.UNSET, spans[0].status.status_code)
assert spans[0].attributes is not None
self.assertTrue(spans[0].attributes['test.websocket_connect_hook'])
self.assertTrue(spans[0].attributes['test.websocket_close_hook'])
if __name__ == '__main__':
unittest.main()
+1 -1
View File
@@ -13,7 +13,7 @@ import aiomcache
from kaya.core import KayaApp, HttpContext
from kaya.session import SessionMixin
from kaya.session_memcache import MemcacheSessionStore
from kaya.session.memcache import MemcacheSessionStore
client = aiomcache.Client('127.0.0.1', 11211)
session = SessionMixin(MemcacheSessionStore(client))
@@ -51,7 +51,7 @@ strict = true
[tool.setuptools_scm]
root = "../.."
version_file = "src/kaya/session_memcache/_version.py"
version_file = "src/kaya/session/memcache/_version.py"
[tool.setuptools_scm.tag]
prefix = "release/"
@@ -7,7 +7,7 @@ from pwo import async_test
from kaya.core import KayaApp, HttpContext
from kaya.session import Session, SessionMixin
from kaya.session_memcache import MemcacheSessionStore
from kaya.session.memcache import MemcacheSessionStore
class FakeClock:
+1 -1
View File
@@ -13,7 +13,7 @@ from redis.asyncio import Redis
from kaya.core import KayaApp, HttpContext
from kaya.session import SessionMixin
from kaya.session_redis import RedisSessionStore
from kaya.session.redis import RedisSessionStore
client = Redis(host='localhost', port=6379)
session = SessionMixin(RedisSessionStore(client))
+1 -1
View File
@@ -51,7 +51,7 @@ strict = true
[tool.setuptools_scm]
root = "../.."
version_file = "src/kaya/session_redis/_version.py"
version_file = "src/kaya/session/redis/_version.py"
[tool.setuptools_scm.tag]
prefix = "release/"
@@ -7,7 +7,7 @@ from pwo import async_test
from kaya.core import KayaApp, HttpContext
from kaya.session import Session, SessionMixin
from kaya.session_redis import RedisSessionStore
from kaya.session.redis import RedisSessionStore
class RedisSessionStoreTest(unittest.TestCase):
@@ -1,3 +1,7 @@
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
from ._mixin import SessionMixin
from ._session import Session
from ._store import InMemorySessionStore, SessionStore
+4
View File
@@ -4,6 +4,10 @@ kaya-session @ file:./packages/kaya-session
kaya-session-redis @ file:./packages/kaya-session-redis
kaya-session-memcache @ file:./packages/kaya-session-memcache
kaya-oidc @ file:./packages/kaya-oidc
kaya-openapi @ file:./packages/kaya-openapi
kaya-cors @ file:./packages/kaya-cors
kaya-forwarded @ file:./packages/kaya-forwarded
kaya-otel @ file:./packages/kaya-otel
build
fakeredis
mypy
+42
View File
@@ -44,6 +44,8 @@ executing==2.2.1
# via stack-data
fakeredis==2.36.2
# via -r requirements-dev.in
googleapis-common-protos==1.75.3
# via opentelemetry-exporter-otlp-proto-http
granian==2.7.9
# via kaya-rsgi
h11==0.16.0
@@ -89,11 +91,23 @@ jeepney==0.9.0
file:./packages/kaya-core
# via
# -r requirements-dev.in
# kaya-cors
# kaya-forwarded
# kaya-oidc
# kaya-openapi
# kaya-otel
# kaya-rsgi
# kaya-session
file:./packages/kaya-cors
# via -r requirements-dev.in
file:./packages/kaya-forwarded
# via -r requirements-dev.in
file:./packages/kaya-oidc
# via -r requirements-dev.in
file:./packages/kaya-openapi
# via -r requirements-dev.in
file:./packages/kaya-otel
# via -r requirements-dev.in
file:./packages/kaya-rsgi
# via -r requirements-dev.in
file:./packages/kaya-session
@@ -126,6 +140,25 @@ mypy-extensions==1.1.0
# via mypy
nh3==0.3.6
# via readme-renderer
opentelemetry-api==1.44.0
# via
# opentelemetry-exporter-otlp-proto-http
# opentelemetry-sdk
# opentelemetry-semantic-conventions
opentelemetry-exporter-otlp-proto-common==1.44.0
# via opentelemetry-exporter-otlp-proto-http
opentelemetry-exporter-otlp-proto-http==1.44.0
# via kaya-otel
opentelemetry-proto==1.44.0
# via
# opentelemetry-exporter-otlp-proto-common
# opentelemetry-exporter-otlp-proto-http
opentelemetry-sdk==1.44.0
# via
# kaya-otel
# opentelemetry-exporter-otlp-proto-http
opentelemetry-semantic-conventions==0.65b0
# via opentelemetry-sdk
packaging==26.2
# via
# build
@@ -138,6 +171,10 @@ pexpect==4.9.0
# via ipython
prompt-toolkit==3.0.52
# via ipython
protobuf==7.36.2
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==7.2.2
# via ipython
ptyprocess==0.7.0
@@ -169,6 +206,7 @@ redis==8.0.1
# kaya-session-redis
requests==2.34.2
# via
# opentelemetry-exporter-otlp-proto-http
# requests-toolbelt
# twine
requests-toolbelt==1.0.0
@@ -193,6 +231,10 @@ typing-extensions==4.16.0
# via
# kaya-core
# mypy
# opentelemetry-api
# opentelemetry-exporter-otlp-proto-http
# opentelemetry-sdk
# opentelemetry-semantic-conventions
# pwo
urllib3==2.7.0
# via