docs_publish
etl_craft.services.docs_publish ¶
publish-docs: serve the catalog site, on this machine or at a link through ngrok.
The site generate-docs writes is served from its folder by a small web server on
127.0.0.1. Each request reads the folder as it is then, so a site written again while it is
served (generate-docs swaps a new one in whole) is served from then on.
Published through ngrok, the site is reachable only by its link: there is no login. Every
response tells search engines and caches to keep out (X-Robots-Tag: noindex, a
robots.txt that disallows everything, Referrer-Policy: no-referrer), no page may be
framed, and folders are never listed. Docs_site.Domain keeps the link the same from one
publish to the next.
Docs_site.Allowed_ips is enforced by this server, whatever the ngrok plan: a visitor from
any other address gets 403. Through ngrok the visitor's address is the last entry of
X-Forwarded-For, which ngrok adds; only ngrok reaches the server on 127.0.0.1, and a
request without that header is refused. Served locally, it is the connecting address.
The URL is recorded in the Engine DB. When a later publish gets another one, links already shared would break, so it fails naming both, unless told to accept the new one.
ROBOTS
module-attribute
¶
Asks every crawler to keep out of the whole site.
HEADERS
module-attribute
¶
HEADERS = {'X-Robots-Tag': 'noindex, nofollow, noarchive', 'Referrer-Policy': 'no-referrer', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'DENY', 'Cache-Control': 'no-cache'}
Sent with every response.
Published
dataclass
¶
Where the site is served, and the local address the web server listens on.
serve ¶
serve(folder: Path, host: str = '127.0.0.1', port: int = 0, *, allowed: tuple[str, ...] = (), behind_tunnel: bool = False) -> ThreadingHTTPServer
Return a web server for the site in folder, not yet serving.
allowed are the CIDR ranges visitors may come from, every address when empty.
ConfigurationError when folder holds no site generate-docs wrote.
published ¶
published(engine: Engine, config: ConnectorConfig, folder: Path, *, local_only: bool = False, host: str = '127.0.0.1', port: int = 0, accept_new_url: bool = False) -> Iterator[Published]
Serve the site for the with body, locally or through ngrok, and yield where.
Through ngrok the URL is checked against the one recorded before and recorded.
open_tunnel ¶
Open an ngrok tunnel to address; return its URL and how to close it.
authtoken ¶
Return the ngrok authtoken from the variable Docs_site.Authtoken names.