GRR Administrator Documentation

GRR supports the ideas of a Security Manager. The Security Manager (Datastore.security_manager config option) handles authorizing those users to resources based on a set of rules.

The default Security Manager is the BasicAccessControlManager. This manager provides rudimentary Admin/Non-Admin functionality, but very little else. See the Auditing section for a discussion on the FullAccessControlManager.

Given the sensitivity of the data that GRR ends up holding, it is important to protect access to it. The AdminUI by default uses basic authentication, based on the passwords within the user objects stored in the data store. However, we support the idea of a Webauth Manager (AdminUI.webauth_manager config option). This allows for customization to use an SSO or SAML based authentication mechanism. However, while we try and make it easy to integrate, this kind of auth is not currently built into GRR.

GRR communication happens using signed and encrypted protobuf messages. We use 1024 bit RSA keys to protect symmetric AES256 encryption. The security of the system does not rely on SSL transport for communication security. This enables easy replacement of the comms protocol with non-http mechanisms such as UDP packets.

The communications use a CA and server public key pair generated on server install. The CA public key is deployed to the client so that it can ensure it is communicating with the correct server. If these keys are not kept secure, anyone with MITM capability can intercept communications and take control of your clients. Additionally, if you lose these keys, you lose the ability to communicate with your clients.

Full details of this protocol and the security properties can be found in the Implementation document.

In addition to the CA and Server key pairs, GRR maintains a set of code signing and driver signing keys. By default GRR aims to provide only read-only actions, this means that GRR is unlikely to modify evidence, and cannot trivially be used to take control of systems running the agent
[Read only access many not give direct code exec, but may well provide it indirectly via read access to important keys and passwords on disk or in memory.]
. However, there are a number of use cases where it makes sense to have GRR execute arbitrary code as explained in the section Deploying custom drivers and code.

However, as part of the GRR design, we decided that administrative control of the GRR server shouldn’t trivially lead to code execution on the clients. As such, we have a separate set of keys for driver signing and code signing. For a driver to be loaded, or binary to be run the code has to be signed by the specific key, the client will confirm this signature before execution.

In the default install, the driver and code signing private keys are not passphrase protected. In a secure environment we strongly recommended generating and storing these keys off the GRR server and doing offline signing every time this functionality is required, or at a minimum setting passphrases which are required on every use.

The open source agent does not contain protection against being disabled by administrator/root on the machine. E.g. on Windows, if an attacker stops the service, the agent will stop and will no longer be reachable. Currently, it is up to the deployer of GRR to provide more protection for the service.

If every deployment in the world is running from the same location and the same code, e.g. c:\program files\grr\grr.exe, it becomes a pretty obvious thing for an attacker to look for and disable. Luckily the attacker has the same problem an investigator has in finding malware on a system, and we can use the same techniques to protect the client. One of the key benefits of having an open architecture is that customization of the client and server is easy, and completely within your control.

For a test, or low security deployment, using the defaults open source agents is fine. However, in a secure environment we strongly recommend using some form of obfuscation.

This can come in many forms, but to give some examples:

GRR does not include any obfuscation mechanisms by default. But we attempt to make this relatively easy by controlling the build process through the configuration file.

In the default setup, clients can register to the GRR server with no prior knowledge. This means that anyone who has a copy of the GRR agent, and knows the address of your GRR server can register their client to your deployment. This significantly eases deployment, and is generally considered low risk as the client has no control or trust on the server. However, it does introduce some risk, in particular: - If there are flows or hunts you deploy to the entire fleet, a malicious client may receive them. These could give away information about what you are searching for. - Clients are allowed to send some limited messages to the server without prompting, these are called Well Known flows. By default these can be used to send log messages, or errors. A malicious client using these could fill up logs and disk space. - If you have custom Well Known Flows that perform interesting actions. You need to be aware that untrusted clients can call them. Most often this could result in a DoS condition, e.g. through a client sending multiple install failure or client crash messages.

In many environments this risk is unwarranted, so we suggest implementing further authorization in the Enrollment Flow using some information that only your client knows, to authenticate it before allowing it to become a registered client.

Note that this does not give someone the ability to overwrite data from another client, as client name collisions are protected.

The http server is designed to be exposed to the Internet, but there is no reason for the other components in the GRR system to be.

The Administration UI by default listens on all interfaces, and is protected by only basic authentication configured via the --htpasswd parameter. We strongly recommend putting the UI on SSL and IP limiting the clients that can connect. The best way to do this normally is by hosting it inside Apache via wsgi, using Apache to provide the SSL and other protection measures.

By default GRR currently only offers limited audit logs in the /var/log/ directory. However, the system is designed to allow for deployment of extensive auditing capabilities through the Security Manager.

The idea is that we have a gateway process, and the Admin UI and any console access is brokered through the gateway. The gateway is the only access to the datastore and it audits all access and can provide intelligent access control. This is implemented in the FullAccessControlManager.

Using this allows for sensible access control, e.g. another user must authorize access before someone is given access to a machine, or an admin must authorize before a hunt is run.

These capabilities are powerful but add significant complexity to deployment so they are not currently released (Mar 2013). If you are interested in this functionality please just email the devs and we can release it.

The AdminUI supports SSL if it is configured. We don’t currently generate certs to enable this by default as certificate management is messy, but you can enable by adding to your config something like:

Note that SSL performance using this method may be average. If you have a lot of users and a single AdminUI, you may get better performance putting GRR behind an SSL reverse proxy such as Apache and letting it handle the SSL.

GRR currently ships with two usable datastores, MongoDB and MySQL, and an extra in memory only test datastore FakeDataStore.

By default GRR will use the MongoDataStore and expect to find it on localhost. You can configure it in the config file, e.g.

We also provide a useful script [scripts/database_reset.sh] to drop the Mongo database which is handy while testing things.

The key issue with performance generally relates to how the server is deployed.

If you are not CPU bound on the individual components (workers, http server) then the key performance differentiator will be the datastore. At the moment we ship with a Mysql and a Mongo datastore.

TBD. User contributions welcome. Using the wsgi hasn’t been thoroughly tested. If you test, please send feedback to the dev list and we can try and fix things.

Changing basic configuration parameters can be done by editing the server config file (/etc/grr/server.local.yaml) to override default values, and then using the config_updater to repack the binaries. This allows for changing basic configuration parameters such as the URL the client reports back to.

Once the config has been edited, you can repack all clients with the new config and upload them to the datastore using grr_config_updater repack_clients

Repacking works by taking the template zip file which are by default installed to /usr/share/grr/executables, injecting relevant configuration files, and renaming files inside the zip to match requested names. This template is then turned into something that can be deployed on the system by using the debian package builder on linux, creating a self extracting zip on Windows, or creating a DMG on OSX.

Doing this is much harder than it should be due to a lot of dependencies and moving parts. It is also a bit of a moving target so we keep the docs for this on the wiki. See:

Configuration of the client is done during the packing/repacking of the client. The process looks like:

When the client runs, it determines the configuration in the following manner based on --config and --secondary_configs arguments that are given to it:

Most parameters are able to be modified by changing parameters and then restarting GRR. However, some configuration options, such as Client.name affect the name of the actual binary itself and therefore can only be changed with a repack on the server.

Updating a configuration variable in the client can be done in multiple ways:

In practice, you should nearly always do 3 or 4.

As an example, to reduce how often the client polls the server to every 300 seconds, you can update the registry as per below, and then restart the service:

The client has numerous configuration parameters that control its behavior, the following explains some key ones you might want to change:

For a full list of available options you can run grr_server --config-help and look for Client, Nanny and Logging options.

To execute an arbitrary python blob, you need to create a file with python code that has the following attributes: - Code in the file must work when executed by exec() in the context of GRR. - Any return data that you want sent back to the server should be stored string encoded in a variable called "magic_return_str".

E.g. as a simple example. The following code modifies the clients poll_max setting and pings test.com.

This file then needs to be signed and converted into the protobuf format required, and then needs to be uploaded to the data store. You can do this using the following command line.

At the end of this you should see something like:

The uploaded files live by convention in aff4:/config/python_hacks and are viewable in the Manage Binaries section of the Admin UI.

The ExecutePythonHack Flow is provided for executing the file on a client.

Drivers are currently used in memory analysis. By default we use drivers developed and released by the GRR team named "pmem". We currently have Apache Licensed, tested drivers for OSX, Linux and Windows.

The drivers are distributed with GRR but are also available from the Volatility project in binary form at http://code.google.com/p/volatility/downloads/list, the source is stored in the scudette branch at http://code.google.com/p/volatility/source/browse/#svn%2Fbranches%2Fscudette%2Ftools.

Deploying a driver works much the same as deploying python code. We sign the file, encode it in a protobuf and upload it to a specific place in the GRR datastore. There is a shortcut to upload the existing memory drivers using config updater.

Deploying this driver would normally be done using the LoadMemoryDriver flow.

The GRR Agent provides an ExecuteBinaryCommand Client Action which allows us to send a binary and set of command line arguments to be executed. The binary must be signed using the executable signing key (config option PrivateKeys.executable_signing_private_key).

To sign an exe for execution use the config updater script.

This uploads to the installers directory by default. But you can override with the --dest_path option.

The client process regularly writes to a registry key (file on Linux and OSX) with a timer. The nanny process watches this registry key called HeartBeat, if it notices that the the client hasn’t updated the heartbeat in the time allocated by UNRESPONSIVE_KILL_PERIOD (default 3 minutes), the nanny will assume the client has hung and will kill it. In Windows we then rely on the Nanny to revive it, on Linux and OSX we rely on the service handling mechanism to do so.

When the client is about to start an action it writes to a registry key containing information about what it is about to do. If the client dies while performing the action, when the client gets restarted it will send an error along with the data from the transaction log to help diagnose the issue.

One tricky thing with the transaction log is the case of Bluescreens or kernel panics. Writing to the transaction log will write a registry key on Windows, but registry keys are not flushed to disk immediately. Therefore, writing a transaction log, and then getting a hard BlueScreen or kernel panic, the transaction log won’t be persistent, and therefore the error won’t be sent. We work around this by adding a Flush to the transaction log when we are about to do dangerous transactions, such as loading a memory driver. But if the client dies during a transaction we didn’t deem as dangerous, it is possible that you will not get a crash report.

We have a hard and a soft memory limit built into the client to stop it getting out of control. The hard limit is enforced by the nanny, if the client goes over that limit it will be hard killed. The soft limit is enforced by the client, if the limit is exceeded the client will stop retrieving new work to do. Once it has finished its current work it will die cleanly.

Default soft limit is 500MB, but GRR should only use about 30MB. Some volatility plugins can use a lot of memory so we try to be generous. Hard limit is double the soft limit. This is configurable from the config file.

A ClientAction can be transmitted from the server with a specified CPU limit, this is how many seconds the action can use. If the action uses more than that it will be killed. The actual implementation is a little more complicated. An action can run for 3 minutes using any CPU it wants before being killed by nanny. However actions that are good citizens (normally the dangerous ones) will call the Progress() function regularly. This function checks if limit has been exceeded and will exit.