README
¶
Grafana Agent is a telemetry collector for sending metrics, logs, and trace data to the opinionated Grafana observability stack. It works best with:
- Grafana Cloud
- Grafana Enterprise Stack
- OSS deployments of Grafana Loki, Prometheus, Grafana Mimir, and Grafana Tempo
Users of Prometheus operating at a massive scale (i.e., millions of active series) can struggle to run an unsharded singleton Prometheus instance: it becomes a single point of failure and requires a giant machine with a lot of resources allocated to it. Even with proper sharding across multiple Prometheus instances, using Prometheus to send data to a cloud vendor can seem redundant: why pay for cloud storage if data is already stored locally?
The Grafana Agent uses the same code as Prometheus, but tackles these issues by only using the most relevant parts of Prometheus for interaction with hosted metrics:
- Service Discovery
- Scraping
- Write Ahead Log (WAL)
- Remote Write
On top of these, the Grafana Agent enables easier sharding mechanisms that enable users to shard Agents across their cluster and lower the memory requirements per machine.
A typical deployment of the Grafana Agent for Prometheus metrics can see up to a 40% reduction in memory usage with equal scrape loads.
The Grafana Agent it can be used to send Prometheus metrics to any system that
supports the Prometheus remote_write API.
Trade-offs
By heavily optimizing Prometheus for remote write and resource reduction, some trade-offs have been made:
- You can't query the Agent; you can only query metrics from the remote write storage.
- Recording rules aren't supported.
- Alerts aren't supported.
- When sharding the Agent, if your node has problems that interrupt metric availability, metrics tracking that node won't be sent for alerting on.
While the Agent can't use recording rules and alerts, remote_write systems such
as Mimir currently support server-side rules and alerts. Note that this trade-off
means that reliability of alerts are tied to the reliability of the remote system
and alerts will be delayed at least by the time it takes for samples to reach
the remote system.
Getting started
When using Kubernetes this link offers the best guide.
Other installation methods can be found in our Grafana Agent documentation.
More detailed documentation is provided as part of the repository.
Example
The example/ folder contains docker-compose configs and a local
k3d/Tanka environment. Both examples deploy the Agent, Cortex and Grafana for
testing the agent. See the docker-compose README
and the k3d example README for more information.
Release schedule
Starting with the v0.28 release, a new minor release is planned for every 6
weeks. This includes a release for Grafana Agent, Grafana Agent Operator, and
the agentctl binary.
The release schedule is best-effort, and releases may be moved forward or backwards if needed. The planned release dates for future minor releases are not changed if one minor release is moved.
Patch and security releases may be created at any time.
Prometheus vendoring
The Grafana Agent vendors a downstream Prometheus repository maintained by Grafana Labs. This is done so experimental features Grafana Labs wants to contribute upstream can first be tested and iterated on quickly within the Agent. We aim to keep the experimental changes to a minimum and upstream changes as soon as possible.
For more context on our vendoring strategy, read our downstream repo maintenance guide.
Getting help
If you have any questions or feedback regarding the Grafana Agent:
- Ask a question on the Agent Slack channel. To invite yourself to the Grafana Slack, visit https://slack.grafana.com/ and join the #agent channel.
- Alternatively ask questions on the Discussions page.
- File an issue for bugs, issues and feature suggestions.
- Attend the Grafana Agent Community Call.
Contributing
Any contributions are welcome and details can be found in our contributors guide.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
agentctl
Command agentctl provides utilities for interacting with Grafana Agent
|
Command agentctl provides utilities for interacting with Grafana Agent |
|
Package component describes the interfaces which Flow components implement.
|
Package component describes the interfaces which Flow components implement. |
|
all
Package all imports all known component packages.
|
Package all imports all known component packages. |
|
common/config
Package config contains types from github.com/prometheus/common/config, but modifiys them to be serializable with River.
|
Package config contains types from github.com/prometheus/common/config, but modifiys them to be serializable with River. |
|
e2e
module
|
|
|
internal
|
|
|
pkg
|
|
|
client
Package client provides a client interface to the Agent HTTP API.
|
Package client provides a client interface to the Agent HTTP API. |
|
client/grafanacloud
Package grafanacloud provides an interface to the Grafana Cloud API.
|
Package grafanacloud provides an interface to the Grafana Cloud API. |
|
cluster
Package cluster enables an agent-wide cluster mechanism which subsystems can use to determine ownership of some key.
|
Package cluster enables an agent-wide cluster mechanism which subsystems can use to determine ownership of some key. |
|
config/features
Package features enables a way to encode enabled features in a flag.FlagSet.
|
Package features enables a way to encode enabled features in a flag.FlagSet. |
|
crow
Package crow implements a correctness checker tool similar to Loki Canary.
|
Package crow implements a correctness checker tool similar to Loki Canary. |
|
flow
Package flow implements the Flow component graph system.
|
Package flow implements the Flow component graph system. |
|
flow/componenttest
Package componenttest provides utilities for testing Flow components.
|
Package componenttest provides utilities for testing Flow components. |
|
integrations/agent
Package agent is an "example" integration that has very little functionality, but is still useful in practice.
|
Package agent is an "example" integration that has very little functionality, but is still useful in practice. |
|
integrations/apache_http
Package apache_http embeds https://github.com/Lusitaniae/apache_exporter
|
Package apache_http embeds https://github.com/Lusitaniae/apache_exporter |
|
integrations/config
Package config provides common configuration structs shared among implementations of integrations.Integration.
|
Package config provides common configuration structs shared among implementations of integrations.Integration. |
|
integrations/consul_exporter
Package consul_exporter embeds https://github.com/prometheus/consul_exporter
|
Package consul_exporter embeds https://github.com/prometheus/consul_exporter |
|
integrations/dnsmasq_exporter
Package dnsmasq_exporter embeds https://github.com/google/dnsmasq_exporter
|
Package dnsmasq_exporter embeds https://github.com/google/dnsmasq_exporter |
|
integrations/elasticsearch_exporter
Package elasticsearch_exporter instantiates the exporter from github.com/justwatchcom/elasticsearch_exporter - replaced for github.com/prometheus-community/elasticsearch_exporter Using the YAML config provided by the agent
|
Package elasticsearch_exporter instantiates the exporter from github.com/justwatchcom/elasticsearch_exporter - replaced for github.com/prometheus-community/elasticsearch_exporter Using the YAML config provided by the agent |
|
integrations/install
Package install registers all in-source integrations for use.
|
Package install registers all in-source integrations for use. |
|
integrations/memcached_exporter
Package memcached_exporter embeds https://github.com/google/memcached_exporter
|
Package memcached_exporter embeds https://github.com/google/memcached_exporter |
|
integrations/mysqld_exporter
Package mysqld_exporter embeds https://github.com/prometheus/mysqld_exporter
|
Package mysqld_exporter embeds https://github.com/prometheus/mysqld_exporter |
|
integrations/postgres_exporter
Package postgres_exporter embeds https://github.com/prometheus/postgres_exporter
|
Package postgres_exporter embeds https://github.com/prometheus/postgres_exporter |
|
integrations/process_exporter
Package process_exporter embeds https://github.com/ncabatoff/process-exporter
|
Package process_exporter embeds https://github.com/ncabatoff/process-exporter |
|
integrations/redis_exporter
Package redis_exporter embeds https://github.com/oliver006/redis_exporter
|
Package redis_exporter embeds https://github.com/oliver006/redis_exporter |
|
integrations/snmp_exporter
Package snmp_exporter embeds https://github.com/prometheus/snmp_exporter
|
Package snmp_exporter embeds https://github.com/prometheus/snmp_exporter |
|
integrations/statsd_exporter
Package statsd_exporter embeds https://github.com/prometheus/statsd_exporter
|
Package statsd_exporter embeds https://github.com/prometheus/statsd_exporter |
|
integrations/v2
Package integrations provides a way to run and manage Grafana Agent "integrations," which integrate some external system (such as MySQL) to Grafana Agent's existing metrics, logging, and tracing subsystems.
|
Package integrations provides a way to run and manage Grafana Agent "integrations," which integrate some external system (such as MySQL) to Grafana Agent's existing metrics, logging, and tracing subsystems. |
|
integrations/v2/agent
Package agent is an "example" integration that has very little functionality, but is still useful in practice.
|
Package agent is an "example" integration that has very little functionality, but is still useful in practice. |
|
integrations/v2/apache_http
Package apache_http embeds https://github.com/Lusitaniae/apache_exporter
|
Package apache_http embeds https://github.com/Lusitaniae/apache_exporter |
|
integrations/v2/autoscrape
Package autoscrape implements a scraper for integrations.
|
Package autoscrape implements a scraper for integrations. |
|
integrations/v2/eventhandler
Package eventhandler watches for Kubernetes Event objects and hands them off to Agent's Logs subsystem (embedded promtail)
|
Package eventhandler watches for Kubernetes Event objects and hands them off to Agent's Logs subsystem (embedded promtail) |
|
integrations/v2/snmp_exporter
Package snmp_exporter embeds https://github.com/prometheus/snmp_exporter
|
Package snmp_exporter embeds https://github.com/prometheus/snmp_exporter |
|
logs
Package logs implements logs support for the Grafana Agent.
|
Package logs implements logs support for the Grafana Agent. |
|
metrics
Package metrics implements a Prometheus-lite client for service discovery, scraping metrics into a WAL, and remote_write.
|
Package metrics implements a Prometheus-lite client for service discovery, scraping metrics into a WAL, and remote_write. |
|
metrics/instance
Package instance provides a mini Prometheus scraper and remote_writer.
|
Package instance provides a mini Prometheus scraper and remote_writer. |
|
metrics/instance/configstore
Package configstore abstracts the concepts of where instance files get retrieved.
|
Package configstore abstracts the concepts of where instance files get retrieved. |
|
operator/assets
Package assets contains helper types used for loading in static assets when configuring the Grafana Agent.
|
Package assets contains helper types used for loading in static assets when configuring the Grafana Agent. |
|
operator/config
Package config generates Grafana Agent configuration based on Kubernetes resources.
|
Package config generates Grafana Agent configuration based on Kubernetes resources. |
|
operator/hierarchy
Package hierarchy provides tools to discover a resource hierarchy.
|
Package hierarchy provides tools to discover a resource hierarchy. |
|
operator/logutil
Package logutil implements an adaptor for the go-kit logger, which is used in the Grafana Agent project, and go-logr, which is used in controller-runtime.
|
Package logutil implements an adaptor for the go-kit logger, which is used in the Grafana Agent project, and go-logr, which is used in controller-runtime. |
|
river/ast
Package ast exposes AST elements used by River.
|
Package ast exposes AST elements used by River. |
|
river/diag
Package diag exposes error types used throughout River and a method to pretty-print them to the screen.
|
Package diag exposes error types used throughout River and a method to pretty-print them to the screen. |
|
river/parser
Package parser implements utilities for parsing River configuration files.
|
Package parser implements utilities for parsing River configuration files. |
|
river/printer
Package printer contains utilities for pretty-printing River ASTs.
|
Package printer contains utilities for pretty-printing River ASTs. |
|
river/scanner
Package scanner implements a lexical scanner for River source files.
|
Package scanner implements a lexical scanner for River source files. |
|
river/token
Package token defines the lexical elements of a River config and utilities surrounding their position.
|
Package token defines the lexical elements of a River config and utilities surrounding their position. |
|
river/token/builder
Package builder exposes an API to create a River configuration file by constructing a set of tokens.
|
Package builder exposes an API to create a River configuration file by constructing a set of tokens. |
|
river/vm
Package vm provides a River expression evaluator.
|
Package vm provides a River expression evaluator. |
|
server
Package server implements the HTTP and gRPC server used throughout Grafana Agent.
|
Package server implements the HTTP and gRPC server used throughout Grafana Agent. |
|
util/k8s
Package k8s spins up a Kubernetes cluster for testing.
|
Package k8s spins up a Kubernetes cluster for testing. |
|
util/structwalk
Package structwalk allows you to "walk" the hierarchy of a struct.
|
Package structwalk allows you to "walk" the hierarchy of a struct. |
|
util/subset
Package subset implements functions to check if one value is a subset of another.
|
Package subset implements functions to check if one value is a subset of another. |
|
tools
|
|
|
crow
Comand grafana-agent-crow is a correctness checker tool that validates that scraped metrics are delivered to a remote_write endpoint.
|
Comand grafana-agent-crow is a correctness checker tool that validates that scraped metrics are delivered to a remote_write endpoint. |
|
rivereval
Command rivereval reads a River file from disk, evaluates it as an expression, and prints the result as a River value.
|
Command rivereval reads a River file from disk, evaluates it as an expression, and prints the result as a River value. |
|
agentlint
Module
|
|
|
web
|
|
|
api
Package api implements the HTTP API used for the Grafana Agent Flow UI.
|
Package api implements the HTTP API used for the Grafana Agent Flow UI. |
|
ui
Package ui exposes utilities to get a Handler for the Grafana Agent Flow UI.
|
Package ui exposes utilities to get a Handler for the Grafana Agent Flow UI. |