[FLINK-39571][docs] Improve Flink Kubernetes Operator documentation coverage

[Dennis-Mircea]

What is the purpose of the change

The operator documentation has grown organically and its structure no longer serves its readers:

• Concepts is an evaluator trap. It opens onto internal controller deep-dives (160 to 325 line developer documents) instead of answering "what does the operator bring me, why install it".
• Custom Resource mixes concerns. The CR structure and reference material is interleaved with operator capabilities such as job management, snapshots, and the autoscaler.
• Operations is a grab-bag. Install-time concerns (Helm, RBAC, configuration) sit next to day-2 concerns (metrics, logging, debugging). Flink core separates these into Deployment and Operations.
• Two custom resources are barely documented (FlinkStateSnapshot) or not at all (FlinkBlueGreenDeployment).
• There is no contributor tier, so internal flows leak into user-facing sections.

This PR restructures the documentation into audience-layered altitudes, following the Flink core documentation convention: the section names, the Try entry point, and the split of install-time (Deployment) from day-2 (Operations) concerns all mirror how Flink core organizes its own documentation, so readers moving between the two find the same structure.

Altitude	Sections	Question answered	Audience
Why	Concepts	Why does this exist, what does it buy me	Evaluators
How to use	Custom Resource, Managing Flink Jobs	How do I run and manage Flink jobs with it	Flink users
How to operate	Deployment, Operations	How do I install the operator and keep it healthy in production	DevOps and platform teams
How it works	Internals	How is it implemented inside	Contributors

On top of the site layering, every capability is told three times, at three depths: Concepts for the why, Managing for the how, Internals for the implementation. For example, snapshots are covered as Concepts → Lifecycle Management → Snapshots (why), Managing → Snapshot Management (how to use), and Internals → Controllers (how the observer and reconciler implement it). Each page links one level down, and every fact has exactly one home with cross-links instead of duplication.

The new information architecture

The numbered bands are the four altitudes from the table above, and the colors carry through the figure: yellow for evaluating the operator, purple for using it day to day, red for operating it in production, gray for contributing to it. The right column shows the resulting sidebar navigation, framed in the color of the band it renders. Reading top to bottom follows the typical reader journey, from first contact through production operation to contribution, and the navigation presents the sections in the same order.

All moved pages keep their old URLs through Hugo aliases, so external links and search results keep working. The alias redirects render for the English tree, matching how the site's aliases have always behaved for the Chinese one.

Brief change log

Information architecture

• New top-level sections Managing Flink Jobs, Deployment, and a rebuilt Internals. Operations is split into install-time (Deployment) and day-2 (Operations) concerns, matching Flink core.
• Capability pages moved out of Custom Resource into Managing, configuration and Helm material moved out of Operations into Deployment, with Hugo aliases preserving every old URL.

Landing and Try

• The landing page is rewritten around a Try It Out / Explore the Documentation / Deploy to Production / Get Help grid.
• The Quick Start is fully rewritten.

Concepts

• Overview rebuilt as a capability map with linked deliverables and capability groups, opening with an at-a-glance deliverables figure.
• New concept pages for Lifecycle Management, Autoscaling, and Zero-Downtime Upgrades, plus a rewritten Architecture page and a Glossary.
• The old controller-flow deep-dives moved to Internals where they belong.

Custom Resource

• Focused on the CR contract: all four kinds (FlinkDeployment, FlinkSessionJob, FlinkStateSnapshot, FlinkBlueGreenDeployment) now have walkthrough examples, relationship and ownership rules, and spec-diffing semantics in one place.
• New Status and Lifecycle page: the full status contract with a lifecycle figure, every status block explained, and a table correlating lifecycleState, reconciliationStatus, jobStatus, and jobManagerDeploymentStatus.

Managing Flink Jobs

• Job Management: validation-gated starting, suspending and resuming, upgrade modes with their snapshot pairing, self-healing (recovery, restarts, rollbacks), and manual recovery.
• Snapshot Management: rebuilt around the FlinkStateSnapshot resource with a state table, lifecycle figure, triggering modes, history, and disposal semantics.
• Blue/Green Deployments: new page for the previously undocumented FlinkBlueGreenDeployment resource, covering deployment states, spec-change behavior, health monitoring, and abort semantics.
• Autoscaler and Autotuning heavily reworked and code-verified, with the configuration knobs presented as key/default/role tables.
• All capability pages close with the same State / Metrics / Events / Limitations sections, and limitations are written consequence-first with their design rationale.

Deployment

• New section: deployment modes overview (including the scale subresource in Standalone mode), Helm subtree (installation, RBAC, cert-manager), operator configuration (with a new Blue/Green configuration section), security (including Flink Web UI exposure), high availability (including JobResultStore cleanup), compatibility, and plugins.

Operations

• New Events catalog and State page (what the operator persists in the cluster, and what backup, cleanup, and GitOps tooling must know about it).
• Metrics and logging split into two focused pages, new Debugging subtree (application profiling, resource exceptions).

Internals

• Collapsed 17 files (10 of them stubs) into five substantive pages: Overview, Startup, Controllers, Webhook, Autoscaler.
• Startup: from the Helm chart materializing the operator to the running JVM, with the configuration model (default, deploy, observe), the informer mappings per controller, and the thread pools with their full concurrency contract.
• Controllers: the complete pipeline at implementation depth, observer requests, the validation rule catalogs, reconciler flows with the upgrade decision table, spec-diff classification, status updates, deletion and cleanup, and snapshot reconciliation, updated against the current implementation rather than copied, with the blue/green state machine traced as a written walkthrough.
• Webhook and Autoscaler: the admission path end to end, and the full derivation of a scaling decision, every request, formula, and branch.
• Written to stay true over time: volatile internal class names are avoided, while resource kinds, spec fields, configuration keys, REST endpoints, and metric names anchor every claim.

Development

• The Guide is rewritten: commands verified against the current build (Java 17, Spotless and Checkstyle, the cert-manager prerequisite for the default-on webhook), the stale IntelliJ shaded-jar workaround removed (the fabric8 relocation it worked around no longer exists), and the IDE section rebuilt around FLINK_CONF_DIR and config.yaml.
• The stale Roadmap page is deleted, JIRA and the mailing lists are the actual planning sources it deferred to.

Editorial conventions

• All pages rewritten in an impersonal voice, no "you" or "we", with the landing page as the single deliberate exception where the welcoming second person matches the tone of the Flink core landing page.
• Consistent conventions applied throughout: American spelling, title-case headings, uniform hint boxes, one link-labeling style, and aligned tables.
• Density pass from the first review round: enumerable content is presented as tables and bullets across Concepts, Custom Resource, and Managing, while narrative prose is kept where pages build an argument.

Figures

• New figure set for concepts, custom-resource, managing, and internals pages (architecture, control loop, lifecycle, autoscaling set, admission control, snapshot lifecycle, CR status and relationships, deliverables board, startup layers), drawn in draw.io with editable sources embedded in the SVGs.
• Legacy class-hierarchy figures kept, and the blue/green state machine is traced as a written walkthrough in Internals.

Chinese documentation

• content.zh is now a verbatim 1:1 mirror of the English tree. The previous zh tree had drifted into a mix of stale translations and English text. A verbatim mirror keeps both trees building, makes the translation debt explicit, and turns future translation work into a clean per-page diff.

Code changes (documentation-supporting only)

• FlinkBlueGreenDeploymentConfigOptions consolidated into KubernetesOperatorConfigOptions with a documented bluegreen section, so the Blue/Green options finally appear in the generated configuration reference. BlueGreenUtils and the controller test updated accordingly.
• CrdReferenceDoclet exclusion list reduced to the classes that must be excluded.
• Two user-facing "behaviour" spellings fixed in javadocs, with the generated config tables and CRD descriptions regenerated. No CRD schema or field changes.

Verifying this change

• Build locally with cd docs && docker run -v $(pwd):/src -p 1313:1313 jakejarvis/hugo-extended:latest server --buildDrafts --buildFuture --bind 0.0.0.0 and browse http://localhost:1313/flink/flink-kubernetes-operator-docs-main/.
• Review the English tree only, content.zh mirrors it verbatim by policy.
• Suggested order: landing page → Concepts → Custom Resource → Managing → Deployment → Operations → Internals, the same path a new user would take.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changes to the CustomResourceDescriptors: no
• Core observer or reconciler logic that is regularly executed: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

Scope cutoff

The content documents the operator as merged to main at the time of writing, the branch is rebased onto it. Open PRs that will change documented behavior are deliberately not pre-documented and will be absorbed once they land, for example:

• [FLINK-40195] helm: use static webhook certificate Secret name #1163 (FLINK-40195): configurable webhook certificate secret name, which touches the webhook TLS and Helm pages.
• [FLINK-39791][helm] Honor defaultConfiguration.config.yaml and always mount config.yaml #1126 (FLINK-39791): honoring defaultConfiguration.config.yaml and always mounting config.yaml, which touches the configuration and startup pages.

Follow-ups

Planned as a continuation of this work, intentionally out of scope here:

• Enrich Try Flink Kubernetes Operator with worked examples and tutorials.
• Update the Autotuning page once the Autotuning Redesign FLIP lands, its Limitations section doubles as the checklist.
• Possible new pages: Production Readiness Checklist and Command Cheatsheet.
• More detail on existing pages, for example concurrency behavior for Flink session clusters and further Blue/Green deployment coverage.
• More diagrams across the sections if needed.
• More depth in Internals if needed.

[gyfora]

Looks good! 👍