webstats

Reads the logs of the node’s web server (nginx or openresty), makes a daily report of who used the sites and what went wrong, and mails it through emailsender.

One yuno per node. Each node reports on its own logs and sends its own mail.

A sealed node has no SSH, so a tool you must log in to run is a task that stops working the day the node is sealed. This one runs inside the node and sends the answer out.

Full design: yunos/c/webstats/README.md

Architecture¶

C_WEBSTATS
    C_TIMER         <- the daily schedule
    C_TIMER         <- the time one whois lookup may take
    C_LOG_READER    <- one per file being read (created, used, destroyed)
    C_PROT_HTTP_CL  <- one per whois lookup (created, used, destroyed)
        C_TCP

A run goes ST_IDLE → ST_READING → ST_LOOKING_UP → ST_REPORTING → ST_IDLE. ST_LOOKING_UP is skipped when there is nothing to look up.

C_LOG_READER turns one file into events (EV_LOG_LINES, EV_LOG_EOF, EV_LOG_ERROR) and knows nothing about nginx.

The two continuations of a run are not timers. A file ends inside the reader’s own publish stack, so the reader cannot be destroyed there, and the work has to cross a cycle of the loop: C_WEBSTATS posts EV_NEXT_FILE to itself and the reader posts EV_READ_CHUNK to itself, with gobj_post_event(). The only timer left is the schedule, which measures a real time. C_WEBSTATS parses the lines, keeps the counters, writes the daily record and hands the mail over.

The line sets the day, not the file¶

The yuno keeps no read offset and does not hook into logrotate. To report day D it reads access.log and access.log.1 and keeps the lines whose own [$time_local] falls inside that day.

So it is idempotent (report-day can be run again), it survives being down for a day, and it does not care when logrotate.timer fires.

What it measures¶

Visitors lead the report. A visitor is an address that asked for a piece of the page (.js or .css) and got it (2xx), and whose user agent carries no crawler mark. A browser fetches the page and its sub-resources. A scanner wearing a browser user agent asks for one URL and leaves. Measured on one node on 2026-08-06: 1346 addresses claimed to be a browser and 71 ever fetched a script.

New visitors are the ones whose fingerprint appears in none of the last new_visitor_days stored days. The record keeps fingerprints, never addresses.

Also: totals and status classes, per hour, per vhost, the top paths / 404s / clients / agents / referrers, every 5xx whole, probes (counted, never banned — that is fail2ban’s job), a latency histogram, and the error log grouped by signature.

Who the top clients are. The first rows of Top clients and Top offenders carry the country, the organisation that holds the network, and the network name, looked up in the registries over RDAP (the JSON successor of whois). See below.

A probe is matched on the percent-decoded path, so the scanner that asks for /%2eenv is counted with the ones that ask for /.env.

Every IPv4 address in the mail is written [a.b.c.d]: OVH’s relay read a bare 34.140.132.132 as a phone number and delivered the mail to nobody. What only separates stays outside ([34.1.2.3]., [34.1.2.3]:443, client:[34.1.2.3], [10.0.0.1]-[10.0.0.9], and an IPv6 address that ends in one is bracketed whole: [::ffff:34.1.2.3], [64:ff9b::34.1.2.3]); an address glued to a word, a slash or a dot is a version and stays as it is (Chrome/142.0.0.0, nginx-1.25.3.1). The stored record keeps the plain address.

Configuration¶

A batch config that turns the lookups off, for a node that cannot reach the registries:

"global": {
    "C_WEBSTATS.whois_enabled": false
}

Who the top clients are¶

One RDAP service answers for every address: RIPE redirects an address it does not hold to the registry that does (8.8.8.8 → ARIN, 1.1.1.1 → APNIC), and the lookup follows the redirect. The row of the record gets:

{"key": "51.38.52.119", "count": 9,
 "whois": {"country": "FR", "org": "OVH SAS", "net": "SD-1G-SBG3-S327B-326B",
           "range": "51.38.52.0 - 51.38.55.255", "source": "rdap.db.ripe.net",
           "looked_up_at": 1790072400}}

and the mail prints 51.38.52.119 FR OVH SAS SD-1G-SBG3-S327B-326B 9.

• The stored days are the cache. An address answered in the last whois_cache_days days is taken from those records and not asked again, so a normal run asks only for the addresses it has not seen this month.

• A lookup never stops the mail. A registry that does not answer within whois_timeout, or refuses, leaves {"error": "timeout"} on the row, a warning in the log and lookup failed: timeout in the mail. A failure is not cached: the next run asks again.

• Private, loopback and link-local addresses are never asked.

• The record says how the names were obtained, and the mail prints it under Sources: "whois": {"enabled": true, "cached": 7, "looked_up": 3, "failed": 0}.

• The node must reach the registries on port 443.

Who fail2ban banned¶

Each row of Top clients and Top offenders also says whether fail2ban banned the address that day, read from /var/log/fail2ban.log and its last rotation:

{"key": "45.148.10.1", "count": 6,
 "banned": {"bans": 1, "at": "10:08", "jails": ["yuneta-nginx-probe"],
            "ban_number": 3, "ban_time": "4d 00:00:00"}}

The mail prints banned 10:08 #3 (4d 00:00:00) or not banned. ban_number and ban_time appear when the escalating bans of tools/fail2ban/install-probe-ban-escalation.sh are installed.

• "banned": false is written only when the log was read. When it could not be (RHEL ships it root:root 0600), the rows say nothing and Needs attention says why; tools/fail2ban/make-fail2ban-log-readable.sh fixes the permissions.

• Needs attention also warns when none of the top offenders that probed 3 times or more was banned: the jail is probably watching nothing.

• A ban lands at the end of a scan (a scan is a few hundred requests in a few seconds), so a scanner appears in Top offenders with its whole count and banned at the same time.

Installing it on a node¶

webstats goes in the utilities batch of the node’s operations repo, next to emailsender and logcenter — the same realm the node’s create-*.sh script builds. That is what makes a node rebuilt from zero come up reporting instead of waiting for somebody to remember it:

{"command": "-install-binary id=webstats content64=$$(webstats)"}
{"command": "-create-config id=webstats.<node> content64=$$(./webstats.<node>.json)"}
{"command": "-create-yuno id=3 realm_id=<utilities realm> yuno_role=webstats yuno_name=<node> must_play=1 yuno_tag=util"}

The binary comes from the package: the .deb and the .rpm ship outputs/yunos/ whole, so install-binary finds it even on a node that carries no SDK sources.

Name the tree the node really serves with. The default reads both the nginx and the openresty tree. A node runs one of them, and the other is usually a leftover whose rotated files are read whole every day to contribute nothing — on one node that was 111000 lines a day for zero rows. Set access_log_paths and error_log_paths to the live tree.

Commands¶

The day parameter is report_date and not date: command-yuno uses its whole kw to select the yuno, so a parameter named like a field of the yuno record matches no yuno and answers “Yuno not found”.

Persistence¶

One TimeRanger2 topic, daily_stats, keyed by the date. Only the aggregates — the rotated .gz files are the archive. A day reported twice keeps both records and the newest answers, which is what makes report-day repeatable.

A run that reads nothing does not replace a run that read something: it is abandoned with a warning. Without that, rebuilding a day whose log already rotated away overwrites a good record with an empty one.

Debugging¶

Enable with ycommand command-yuno id=<id> service=__yuno__ command=set-gclass-trace gclass=C_WEBSTATS set=1 level=parse.

parse is the one that matters: it is the only way to see a log format change that the parser silently tolerates.