Publish EventLens v1.1.0 (#1)

This commit was merged in pull request #1.
This commit is contained in:
Jens committed 2026-11-20 12:20:18 +00:00
1 parent c0ade21029
commit 87432f3283
63 files changed
+7381 -1420

No files matched your search

+3 -3
View File
@@ -16,7 +16,7 @@ normalized events (bounded strings/fields, explicit timestamp validity/source, s
deterministic correlation rules -> incidents + reasons + evidence IDs
|
v
React workspace shell (Analysis/Timeline/Hunting/Reports/Settings) -> virtualized timeline/filter/inspector -> cloned redacted JSON/Markdown export
React workspace shell (Analysis/Timeline/Hunting/Reports/Settings) -> virtualized timeline/filter/inspector -> worker-redacted JSON/Markdown export
```
## Boundaries
@@ -24,7 +24,7 @@ React workspace shell (Analysis/Timeline/Hunting/Reports/Settings) -> virtualize
- `backend/app/main.py` owns HTTP, CORS, receive-level request limits, bounded parse admission, privacy headers, upload reading and response contracts.
- `backend/app/parser.py` owns the untrusted EVTX boundary and never emits raw XML to presentation code.
- `backend/app/normalize.py` maps parser-native JSON into `NormalizedEvent` values, with bounded fields, UTC timestamps and explicit `event`/`record`/`missing`/`invalid` timestamp metadata.
- `backend/app/correlate.py` consumes normalized events only. It never parses raw EVTX, requires provider/channel context for provider-scoped IDs, avoids unknown grouping keys and attaches stable rule/incident/evidence identifiers.
- `backend/app/correlate.py` consumes normalized events only. It never parses raw EVTX, requires provider/channel context for provider-scoped IDs, avoids unknown grouping keys and attaches stable rule/incident/evidence identifiers. Campaigns summarize adjacent incident windows for the same rule and grouping key.
- `backend/app/models.py` defines the API contract.
- `frontend/src/filter.ts`, `redact.ts` and `export.ts` are pure browser-side workflow functions.
- `frontend/src/main.tsx` is only the React bootstrap. `frontend/src/app/App.tsx` owns explicit workspace state (active hash workspace, selection, filters, settings and loading/error state); reusable shell/state components live in `frontend/src/components/`, and Analysis, Timeline, Hunting, Reports and Settings live in `frontend/src/workspaces/` over the same in-memory `Analysis` object. The browser UI does not implement parser or correlation rules.
@@ -35,6 +35,6 @@ Uploads are bounded while ASGI body messages are received, then read in bounded
API and health responses are non-cacheable and carry defensive browser headers. The production container runs as an unprivileged user with a read-only root filesystem, dropped capabilities and a bounded temporary filesystem. The browser keeps only the current normalized analysis in memory; large timelines are virtualized and search documents are cached per event.
The deployment broker passes the validated main commit as `GITEA_COMMIT_SHA` into the Compose build. Docker embeds that exact SHA as `EVENTLENS_REVISION`; `/health` exposes it with the product version without running Git in the container. An unlabelled local build reports `local`.
Docker accepts the source SHA as `EVENTLENS_REVISION` and records it in the OCI revision label. The managed deployment also supplies the exact revision through its environment. `/health` exposes the revision and product version without running Git in the container. An unlabelled local build reports `local`.
The browser stores only one validated local-preferences record in `localStorage` (`eventlens.preferences.v1`); it never stores events, analysis results or EVTX-derived values. Settings writes that record automatically, exposes an accessible save confirmation and can reset it to validated defaults. A configured default workspace is used only when no URL hash is present, so a hash/deep link always wins. Workspace navigation uses a lightweight URL hash so browser back/forward works without a routing dependency. The static browser-brand package (SVG/ICO/PNG/manifest) is bundled locally, has no service worker and does not add a cache or network dependency. The bundled JSON demo exercises the same normalized, correlation and presentation contracts while deliberately skipping only raw EVTX parsing. The real parser is qualified in CI against a public, hash-verified seven-record EVTX fixture without committing that fixture or its event data to the repository.
+25 -1
View File
@@ -13,17 +13,41 @@ Every incident carries a stable `rule_id`, bounded total time window, explicit p
## Rules and boundaries
- `windows.audit.log-cleared` — EventLog 1102 (Security audit log cleared) from the Security channel or EventLog 104 (an event log was cleared) from the System channel, in a bounded 300-second window per known host. Severity `critical`: records from before the clear may be missing. The reason states only what the record says about who cleared it.
- `windows.crash.application-report` — Application Error 1000 and/or Windows Error Reporting 1001 from the exact Application provider/channel within 120 seconds, grouped by known host/process. The word “sequence” is reserved for evidence containing both records.
- `windows.service.failure` — abnormal Service Control Manager 7023/7024/7031/7034 events from the System channel within 120 seconds, grouped by known host/service.
- `windows.service.installed` — Service Control Manager 7045 from the System channel in a bounded 300-second window per known host/service name. Installation alone is not evidence of malice.
- `windows.network.dns-timeout` — DNS Client 1014 events from the System channel in a bounded 120-second window per known host.
- `windows.auth.failed-logon` — Security Auditing 4625 events from the Security channel in a bounded 120-second window per known host/account. One event is recorded evidence; two are repeated failures; three or more are called a burst.
- `windows.auth.password-spray` — Security Auditing 4625 in a bounded 300-second window per known host and source address (`IpAddress`), kept only when the window contains at least three distinct target accounts. The reason says the pattern is consistent with, not proof of, password spraying. These events also remain evidence for `windows.auth.failed-logon`.
- `windows.auth.account-lockout` — Security Auditing 4740 in a bounded 300-second window per known host/account.
- `windows.account.change` — Security Auditing 4720 (user created) and 4728/4732/4756 (member added to a security group) in a bounded 300-second window per known host.
- `windows.system.startup` — Kernel-General 12 or EventLog 6005 startup signals in a bounded 300-second window per known host.
- `windows.system.shutdown.normal` — Kernel-General 13 or EventLog 6006 normal-shutdown evidence in a bounded 300-second window per known host.
- `windows.system.shutdown.unexpected` — Kernel-Power 41 or EventLog 6008 unexpected-shutdown/restart evidence in a bounded 300-second window per known host. The reason describes evidence, not a cause.
- `windows.generic.attention` — preserves unmatched warning/error evidence rather than hiding it.
- `windows.generic.attention` — preserves unmatched warning/error evidence rather than hiding it. Unmatched events with the same known host, provider and event ID are burst-compressed into one bounded 300-second window; the incident keeps the highest level (`critical`, `error` or `warning`) of its evidence. Events with an unknown host, provider or event ID, or without a reliable timestamp, stay single-event incidents.
A rule requires its provider and channel context; a coincidental Event ID from another provider is not classified as that rule. Missing/placeholder host, process, service or account values create an event-isolated group and the reason says that grouping was unavailable; they never become a shared `unknown-*` key. Events without a reliable timestamp remain evidence but do not enter temporal groups.
A group is split when adding the next event would make the total span from the first event exceed the rule window. Equality at the boundary is included; one millisecond beyond it is split. This prevents a long chain of near-adjacent records from creating an unbounded incident.
Repeated identical provider/event/summary signatures remain in the evidence timeline but are compressed in the incident summary; source records and evidence IDs are never removed. Events may support more than one specifically matching rule when their predicates independently match; no rule is treated as proof of root cause.
## Campaigns
Bounded windows keep incidents honest but fragment sustained activity: a brute force that runs for hours becomes thousands of 120-second incidents. A **campaign** is a summary layer on top of incidents, never a wider incident:
- it links consecutive incidents of the same rule and the same known grouping key when the gap between one incident's last event and the next incident's first event is at most the rule window;
- it needs at least two incidents; isolated (unknown-key) and untimed incidents never join one;
- it carries a deterministic `campaign-…` ID derived from the rule and the ordered incident IDs, the time span, the incident IDs, the total evidence count, the highest severity and a reason that makes no causal claim;
- each member incident references it through `campaign_id`. Incident windows, evidence IDs and incident IDs are unchanged.
The Analysis workspace shows campaigns first and pages the remaining incidents by severity, so a large log stays readable.
## Audit outcome
Security events are logged at level 0 (“LogAlways”); success or failure lives in the `Keywords` bitmask. EventLens normalizes the audit-failure (`0x0010000000000000`) and audit-success (`0x0020000000000000`) bits into `outcome: "failure" | "success"`. The informational-noise filter treats audit failures as attention events, so failed sign-ins are not hidden by default.
## Source integrity
For uploaded files the API returns `source_info` with the SHA-256 and byte size of the exact uploaded bytes, computed in memory before parsing. Reports and exports include it so an analysis can be tied to the original file. The bundled demo has no `source_info`.
+5 -2
View File
@@ -19,6 +19,7 @@ Redaction is enabled by default and operates on a cloned analysis object. It mas
- computer/host values;
- username/account/e-mail identity fields, including domain-qualified accounts, SIDs and machine accounts;
- domain-name fields (`SubjectDomainName`, `TargetDomainName`), DNS query names and fully qualified host names with three or more labels in free text (two-label names such as `app.exe` are left alone);
- e-mail addresses in free text;
- IPv4 and IPv6 addresses in free text or IP-address fields;
- Windows and UNC paths (including spaces/parentheses) and path-like fields;
@@ -26,12 +27,14 @@ Redaction is enabled by default and operates on a cloned analysis object. It mas
- known structured identities propagated into summaries, incident titles and reasons;
- the source filename in the exported report.
The UI previews replacement counts by type before export. The original in-memory analysis is never modified by the redaction function. The source name is replaced with `[REDACTED_SOURCE].evtx` when redaction is enabled. JSON and Markdown carry the same redacted normalized analysis and parse-completeness metadata.
Replacements are consistent pseudonyms: the same value always becomes the same numbered token (`[REDACTED_HOST_1]`, `[REDACTED_IDENTITY_3]`), so an exported report still shows which events share a host or account without revealing it. The UI previews replacement counts by type before export and lists the field names whose values were exported without masking, so a reviewer can check them explicitly.
Redaction runs in a Web Worker when Reports or an export needs it, never on load, so large analyses do not block the page. If the worker cannot run, the same function runs in the page; if redaction fails, export stays disabled rather than falling back to raw data. The original in-memory analysis is never modified by the redaction function. The source name is replaced with `[REDACTED_SOURCE].evtx` when redaction is enabled. JSON and Markdown carry the same redacted normalized analysis, campaigns and parse-completeness metadata. The source SHA-256 is kept in redacted exports: it identifies the file, not a person, and lets the recipient verify which evidence was analyzed.
Redaction can be disabled when a raw forensic export is explicitly required, but that is an explicit user choice and the warning remains visible.
## Limits
The API enforces a 50 MiB file limit, a bounded request body independent of `Content-Length`, parser event/time budgets, bounded normalized strings/fields/output and bounded parse concurrency. These controls reduce accidental resource exhaustion; they are not a guarantee against every parser implementation vulnerability.
The API admits an upload (up to the parse concurrency plus a small queue) before the multipart body is spooled to the in-memory `/tmp` tmpfs, so a burst of large uploads gets a fast `503` instead of exhausting it. The API enforces a 50 MiB file limit, a bounded request body independent of `Content-Length`, parser event/time budgets, bounded normalized strings/fields/output and bounded parse concurrency. These controls reduce accidental resource exhaustion; they are not a guarantee against every parser implementation vulnerability.
Redaction is a best-effort privacy layer, not a formal DLP engine. Application-specific free-form values may contain sensitive data that cannot be inferred safely. Review exports before sharing them outside the intended context.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 279 KiB

After

Width:  |  Height:  |  Size: 659 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 260 KiB

After

Width:  |  Height:  |  Size: 291 KiB