koszewscy/python-helpers
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add initial implementation of Python helpers and CLI tools

Browse Source 
- Create .gitignore to exclude build artifacts
- Add README.md with project description and installation instructions
- Implement pyproject.toml for package metadata and script entry points
- Add console output helpers for command-line scripts
- Implement environment file loading functionality
- Add JSON file I/O helpers
- Create token decoder CLI for decoding JWT claims
- Implement serve-markdown CLI for serving Markdown files as HTML
This commit is contained in:
slawek
2026-09-06 19:15:54 +02:00
commit 9f56b3ee19
10 changed files with 629 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+4
View File
@@ -0,0 +1,4 @@
__pycache__/
*.egg-info/
dist/
build/
README.md
+82
View File
@@ -0,0 +1,82 @@
# Python Helpers
Generic Python helpers and command-line tools with no cloud-specific logic. It has no external
dependencies.
- `serve-markdown`: serve Markdown files as live-reloading GitHub-styled HTML pages
- `token-decoder`: decode a JWT's claims without verifying its signature
## Installation
Each command below is available on `PATH` directly (no `.py` extension, no `python3` prefix)
once the package is installed into the active virtual environment, by any of the methods below.
### Local install
From a checkout of this repository:
```bash
pip install -e . # editable, for development: picks up source changes without reinstalling
pip install . # normal install: copies the package in
```
### Install from git
```bash
pip install git+<repository-url>@main
```
### Running without installing
From a repo checkout, run any command as a module with `-m` instead of installing the package:
```bash
python3 -m python_helpers.cli.token_decoder --help
```
From elsewhere, set `PYTHONPATH` to the repo root instead:
```bash
PYTHONPATH=/path/to/python-helpers python3 -m python_helpers.cli.token_decoder --help
```
The module path replaces the installed command's short hyphenated name; the underlying `main()`
and behavior are identical.
## Library
- `console`: colored console output and the `fail()` helper used to report errors and exit
- `env`: loading docker-style `NAME=VALUE` environment files into the process environment
- `files`: JSON file load/save helpers
## Commands
### `token-decoder`
Decode a JWT's claims, without verifying its signature. Reads from a file argument, or stdin.
Example:
```bash
token-decoder token.txt
echo "$TOKEN" | token-decoder
```
### `serve-markdown`
Render Markdown files under a directory as GitHub-styled pages and serve them locally, live-reloading
on change.
Parameters:
- `path` (optional, positional): Markdown file or directory to serve; defaults to the current directory. A directory is served at its own URL path, resolving to `index.md` or `README.md` within it. Prefix with `pydoc:` and a dotted module name (e.g. `pydoc:python_helpers.console`) to render that module's docs with `pydoc-markdown` instead; requires the `pydoc-markdown` package to be installed
- `--listen-address` (optional): Address for the local web server to listen on (default: `127.0.0.1`)
- `--port` (optional): Port for the local web server (default: `8000`)
- `--watch-interval` (optional): Seconds between checks for changes to the file, polled by the browser page (default: `1.0`)
Example:
```bash
serve-markdown README.md
serve-markdown pydoc:python_helpers.console
```
pyproject.toml
+16
View File
@@ -0,0 +1,16 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "python-helpers"
version = "1.0.0"
description = "Generic Python helpers and command-line tools with no cloud-specific logic"
requires-python = ">=3.9"
[project.scripts]
serve-markdown = "python_helpers.cli.serve_markdown:main"
token-decoder = "python_helpers.cli.token_decoder:main"
[tool.setuptools.packages.find]
include = ["python_helpers*"]
python_helpers/__init__.py
+1
View File
@@ -0,0 +1 @@
"""python_helpers - generic Python helpers with no cloud-specific logic."""
python_helpers/cli/__init__.py
+1
View File
@@ -0,0 +1 @@
"""python_helpers.cli - console-script entry points for the generic commands."""
python_helpers/cli/serve_markdown.py
Executable
+335
View File
@@ -0,0 +1,335 @@
#!/usr/bin/env python3
"""Serve Markdown files under a directory as GitHub-styled HTML pages over a local web server."""
import argparse
import html
import http.server
import json
import sys
import time
import urllib.parse
from pathlib import Path
from python_helpers.console import COLOR_GREEN, COLOR_RESET, COLOR_YELLOW
GITHUB_MARKDOWN_CSS_URL = "https://cdn.jsdelivr.net/npm/github-markdown-css@5/github-markdown.min.css"
MARKED_JS_URL = "https://cdn.jsdelivr.net/npm/marked@15/marked.min.js"
DOMPURIFY_JS_URL = "https://cdn.jsdelivr.net/npm/dompurify@3/dist/purify.min.js"
HIGHLIGHT_JS_URL = "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"
HIGHLIGHT_CSS_LIGHT_URL = "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/github.min.css"
HIGHLIGHT_CSS_DARK_URL = "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/github-dark.min.css"
DEFAULT_LISTEN_ADDRESS = "127.0.0.1"
DEFAULT_SERVE_PORT = 8000
DEFAULT_CANDIDATES = ["index.md", "README.md"]
DEFAULT_WATCH_INTERVAL_SECONDS = 1.0
PYDOC_PREFIX = "pydoc:"
class MarkdownServeError(Exception):
"""A user-facing error."""
def fail(message):
"""Raise a MarkdownServeError with message.
Args:
message: error text.
"""
raise MarkdownServeError(message)
def find_index_file(directory):
"""Return the first of DEFAULT_CANDIDATES that exists as a file under directory, or None.
Args:
directory: directory to look in.
"""
return next(
(path for path in (directory / candidate for candidate in DEFAULT_CANDIDATES) if path.is_file()),
None,
)
def resolve_route(route, root_dir, default_file):
"""Resolve a URL route to a Markdown file under root_dir, or None if it cannot be served.
Args:
route: URL path, e.g. "/", "/mypage.md", "/hello/", "/hello/notes.md".
root_dir: directory routes are resolved relative to.
default_file: file to use for the "/" route, or None to fall back to DEFAULT_CANDIDATES.
"""
relative = urllib.parse.unquote(route).lstrip("/")
if relative == "":
return default_file if default_file is not None else find_index_file(root_dir)
if relative.endswith("/"):
return find_index_file(root_dir / relative)
candidate = root_dir / relative
try:
resolved = candidate.resolve()
resolved.relative_to(root_dir.resolve())
except ValueError:
return None
return resolved if resolved.is_file() else None
def render_pydoc_markdown(module_name):
"""Render a dotted Python module name to a Markdown string using pydoc-markdown.
Args:
module_name: dotted module name, e.g. "cloud_tools.az".
"""
try:
from pydoc_markdown import PydocMarkdown
from pydoc_markdown.contrib.loaders.python import PythonLoader
except ImportError:
fail("pydoc-markdown is not installed; install it to use pydoc: module paths")
session = PydocMarkdown()
session.loaders = [PythonLoader(modules=[module_name])]
modules = session.load_modules()
session.process(modules)
return session.renderer.render_to_string(modules)
def render_page(title, content_query, watch_interval_ms):
"""Build the HTML page shell that polls /content and renders it as Markdown.
Args:
title: page title.
content_query: query string (including leading '?') appended to the /content request.
watch_interval_ms: milliseconds between polls of /content.
"""
body = f"""<article id="content" class="markdown-body">Loading...</article>
<script>
let lastMtime = null;
function render(text) {{
const target = document.getElementById("content");
if (window.marked && window.DOMPurify) {{
target.innerHTML = DOMPurify.sanitize(marked.parse(text, {{gfm: true}}));
if (window.hljs) {{
target.querySelectorAll("pre code").forEach((block) => hljs.highlightElement(block));
}}
}} else {{
const pre = document.createElement("pre");
pre.textContent = text;
target.innerHTML = "";
target.appendChild(pre);
}}
}}
async function poll() {{
let data;
try {{
data = await (await fetch("/content{content_query}")).json();
}} catch (error) {{
return;
}}
if (data.mtime !== null && data.mtime !== lastMtime) {{
lastMtime = data.mtime;
render(data.text);
}} else if (data.mtime === null) {{
render("(file not found: " + data.error + ")");
}}
}}
poll();
setInterval(poll, {watch_interval_ms});
</script>"""
return f"""<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{html.escape(title)}</title>
<link rel="stylesheet" href="{GITHUB_MARKDOWN_CSS_URL}">
<link rel="stylesheet" href="{HIGHLIGHT_CSS_LIGHT_URL}" media="(prefers-color-scheme: light)">
<link rel="stylesheet" href="{HIGHLIGHT_CSS_DARK_URL}" media="(prefers-color-scheme: dark)">
<script src="{MARKED_JS_URL}"></script>
<script src="{DOMPURIFY_JS_URL}"></script>
<script src="{HIGHLIGHT_JS_URL}"></script>
<style>
body {{ margin: 0; background-color: #ffffff; }}
@media (prefers-color-scheme: dark) {{
body {{ background-color: #0d1117; }}
}}
.markdown-body {{ box-sizing: border-box; max-width: 980px; margin: 0 auto; padding: 45px; }}
.markdown-body pre {{ white-space: pre; overflow-x: auto; }}
</style>
</head>
<body>
{body}
</body>
</html>
""".encode("utf-8")
class MarkdownRequestHandler(http.server.BaseHTTPRequestHandler):
"""Serves a page shell for any Markdown file under root_dir, and its live content at /content.
root_dir, default_file, and watch_interval_ms are set as class attributes before the server
starts, since HTTPServer instantiates request handlers itself with a fixed signature.
"""
root_dir = None
default_file = None
pydoc_module = None
watch_interval_ms = 0
def send_json(self, payload):
content_bytes = json.dumps(payload).encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(content_bytes)))
self.end_headers()
self.wfile.write(content_bytes)
def send_not_found(self, message):
body = message.encode("utf-8")
self.send_response(404)
self.send_header("Content-Type", "text/plain; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self):
parsed = urllib.parse.urlsplit(self.path)
if self.pydoc_module is not None:
if parsed.path == "/content":
self.send_json({"mtime": time.time(), "text": render_pydoc_markdown(self.pydoc_module), "error": None})
return
content_query = "?" + urllib.parse.urlencode({"path": parsed.path})
page_bytes = render_page(self.pydoc_module, content_query, self.watch_interval_ms)
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(page_bytes)))
self.end_headers()
self.wfile.write(page_bytes)
return
if parsed.path == "/content":
query = urllib.parse.parse_qs(parsed.query)
route = query.get("path", [""])[0]
markdown_path = resolve_route(route, self.root_dir, self.default_file)
if markdown_path is None:
self.send_json({"mtime": None, "text": None, "error": f"no such file for route '{route}'"})
return
try:
mtime = markdown_path.stat().st_mtime
text = markdown_path.read_text(encoding="utf-8")
self.send_json({"mtime": mtime, "text": text, "error": None})
except OSError as error:
self.send_json({"mtime": None, "text": None, "error": str(error)})
return
markdown_path = resolve_route(parsed.path, self.root_dir, self.default_file)
if markdown_path is None:
self.send_not_found(f"no Markdown file found for '{parsed.path}'")
return
content_query = "?" + urllib.parse.urlencode({"path": parsed.path})
page_bytes = render_page(markdown_path.name, content_query, self.watch_interval_ms)
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(page_bytes)))
self.end_headers()
self.wfile.write(page_bytes)
def log_message(self, format_str, *log_args):
pass
def main():
"""CLI entry point: parse args, resolve the document root, and serve Markdown files under it."""
parser = argparse.ArgumentParser(
description="Render Markdown files under a directory as GitHub-styled pages and serve them locally."
)
parser.add_argument(
"path",
nargs="?",
help=(
"Markdown file or directory to serve; defaults to the current directory. "
"A directory is served at its own URL path, resolving to index.md or README.md within it. "
f"Prefix with '{PYDOC_PREFIX}' and a dotted module name (e.g. '{PYDOC_PREFIX}cloud_tools.az') "
"to render that module's docs with pydoc-markdown instead"
),
)
parser.add_argument(
"--listen-address",
default=DEFAULT_LISTEN_ADDRESS,
help=f"Address for the local web server to listen on (default: {DEFAULT_LISTEN_ADDRESS})",
)
parser.add_argument(
"--port",
type=int,
default=DEFAULT_SERVE_PORT,
help=f"Port for the local web server (default: {DEFAULT_SERVE_PORT})",
)
parser.add_argument(
"--watch-interval",
type=float,
default=DEFAULT_WATCH_INTERVAL_SECONDS,
help=(
"Seconds between checks for changes to the file, polled by the browser page "
f"(default: {DEFAULT_WATCH_INTERVAL_SECONDS})"
),
)
args = parser.parse_args()
try:
default_file = None
pydoc_module = None
source_description = None
if args.path and args.path.startswith(PYDOC_PREFIX):
pydoc_module = args.path[len(PYDOC_PREFIX):]
if not pydoc_module:
fail("pydoc: requires a module name, e.g. 'pydoc:cloud_tools.az'")
render_pydoc_markdown(pydoc_module)
root_dir = None
source_description = f"pydoc module '{pydoc_module}'"
elif args.path:
given_path = Path(args.path)
if given_path.is_file():
root_dir = given_path.parent
default_file = given_path
elif given_path.is_dir():
root_dir = given_path
else:
fail(f"'{given_path}' does not exist or is not a file or directory")
else:
root_dir = Path.cwd()
if pydoc_module is None:
if default_file is None and find_index_file(root_dir) is None:
fail(f"no file given and none of {', '.join(DEFAULT_CANDIDATES)} found in '{root_dir}'")
source_description = f"'{root_dir}'"
watch_interval_ms = round(args.watch_interval * 1000)
MarkdownRequestHandler.root_dir = root_dir
MarkdownRequestHandler.default_file = default_file
MarkdownRequestHandler.pydoc_module = pydoc_module
MarkdownRequestHandler.watch_interval_ms = watch_interval_ms
try:
server = http.server.HTTPServer((args.listen_address, args.port), MarkdownRequestHandler)
except OSError as error:
fail(f"could not start server on {args.listen_address}:{args.port} ({error}); try a different --listen-address/--port")
url = f"http://{args.listen_address}:{args.port}/"
print(f"{COLOR_GREEN}Serving at {url} (Ctrl+C to stop){COLOR_RESET}")
print(f"{COLOR_GREEN}Document root: {source_description}, watching for changes every {args.watch_interval}s{COLOR_RESET}")
try:
server.serve_forever()
except KeyboardInterrupt:
print()
print(f"{COLOR_YELLOW}Stopping server{COLOR_RESET}")
finally:
server.server_close()
except MarkdownServeError as error:
print(f"Error: {error}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
python_helpers/cli/token_decoder.py
Executable
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env python3
"""Decode a JWT's claims, without verifying its signature."""
import argparse
import base64
import json
import sys
from python_helpers.console import fail
EXAMPLES = """
Examples:
%(prog)s token.txt
echo "$TOKEN" | %(prog)s
This command will:
1. Read a JWT from the given file, or from stdin if no file is given
2. Decode its payload claims, without verifying its signature
3. Print the claims as JSON
"""
def decode_jwt_claims(token):
"""Decode a JWT's payload claims, without verifying its signature.
Args:
token: a JSON Web Token (JWT) string.
Returns:
The decoded claims as a dict.
"""
payload = token.split(".")[1]
payload += "=" * (-len(payload) % 4)
return json.loads(base64.urlsafe_b64decode(payload))
def build_parser():
"""Build the argparse parser for this script."""
parser = argparse.ArgumentParser(
description="Decode a JWT's claims, without verifying its signature.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=EXAMPLES,
)
parser.add_argument(
"token_file",
nargs="?",
help="Path to a file containing a JWT; reads from stdin if omitted",
)
return parser
def main():
"""Parse arguments and print the decoded claims of the given (or piped-in) token."""
args = build_parser().parse_args()
if args.token_file:
try:
with open(args.token_file, encoding="utf-8") as token_file:
token = token_file.read().strip()
except OSError as error:
fail(f"Could not read '{args.token_file}': {error}")
else:
token = sys.stdin.read().strip()
if not token:
fail("No token given")
print(json.dumps(decode_jwt_claims(token), indent=2))
if __name__ == "__main__":
main()
python_helpers/console.py
+47
View File
@@ -0,0 +1,47 @@
"""Colored console output helpers for command-line scripts."""
import sys
if sys.stdout.isatty():
COLOR_RESET = "\033[0m"
COLOR_RED = "\033[0;31m"
COLOR_GREEN = "\033[0;32m"
COLOR_YELLOW = "\033[0;33m"
COLOR_BLUE = "\033[0;34m"
COLOR_CYAN = "\033[0;36m"
else:
COLOR_RESET = COLOR_RED = COLOR_GREEN = COLOR_YELLOW = COLOR_BLUE = COLOR_CYAN = ""
def fail(message):
"""Print an error message to stderr and exit with status 1.
Args:
message: error text to print.
"""
print(f"{COLOR_RED}Error: {message}{COLOR_RESET}", file=sys.stderr)
sys.exit(1)
def print_banner(title, title_color=COLOR_CYAN):
"""Print a cyan banner line, the given title, and another banner line.
Args:
title: text to print between the banner lines.
title_color: color to print the title in; the banner lines themselves are always
cyan. Defaults to cyan, matching most banners; pass COLOR_GREEN for a
"Setup/Cleanup Complete!" style closing banner.
"""
print(f"{COLOR_CYAN}=========================================={COLOR_RESET}")
print(f"{title_color}{title}{COLOR_RESET}")
print(f"{COLOR_CYAN}=========================================={COLOR_RESET}")
def print_summary_counts(pairs):
"""Print a flat "Label: count" line for each given (label, count) pair.
Args:
pairs: iterable of (label, count) tuples.
"""
for label, count in pairs:
print(f"{label}: {count}")
python_helpers/env.py
+33
View File
@@ -0,0 +1,33 @@
"""Environment file helpers for command-line scripts."""
import os
from python_helpers.console import fail
def load_env_file(path):
"""Load a docker-style environment file into the process environment.
Blank lines and lines whose first non-blank character is # are ignored; a # anywhere
else is part of the value. A NAME=VALUE line sets the variable, overriding any value
already in the environment; a bare NAME line leaves the inherited value in place.
Values are taken literally, without quote removal or variable interpolation. An empty
name, or one containing whitespace, is an error.
Args:
path: path to the environment file.
"""
try:
with open(path, encoding="utf-8") as env_file:
lines = env_file.read().splitlines()
except OSError as error:
fail(f"Could not read '{path}': {error}")
for number, line in enumerate(lines, start=1):
if not line.strip() or line.lstrip().startswith("#"):
continue
name, separator, value = line.partition("=")
if not name or any(character.isspace() for character in name):
fail(f"{path}: line {number}: variable '{name}' is empty or contains whitespace")
if separator:
os.environ[name] = value
python_helpers/files.py
+37
View File
@@ -0,0 +1,37 @@
"""JSON file I/O helpers for command-line scripts."""
import json
from pathlib import Path
from python_helpers.console import fail
def load_json(path, missing_hint=None):
"""Load and parse a JSON file, failing with a consistent message if it is missing.
Args:
path: path to the JSON file.
missing_hint: optional extra line of guidance to print if the file is missing.
Returns:
The parsed JSON content.
"""
if not Path(path).exists():
message = f"{path} not found!"
if missing_hint:
message += f"\n{missing_hint}"
fail(message)
with open(path, "r") as f:
return json.load(f)
def save_json(path, data, indent=2):
"""Save data to a file as JSON.
Args:
path: path to write to.
data: JSON-serializable data to write.
indent: indentation level to format the JSON with.
"""
with open(path, "w") as f:
json.dump(data, f, indent=indent)