improved documentation
This commit is contained in:
141
README.md
141
README.md
@@ -1,4 +1,44 @@
|
|||||||
# Remote Build Cache Server
|
# Remote Build Cache Server
|
||||||
|
|
||||||
|

|
||||||
|

|
||||||
|

|
||||||
|

|
||||||
|
|
||||||
|
<!--
|
||||||
|

|
||||||
|
-->
|
||||||
|
|
||||||
|
Speed up your builds by sharing and reusing unchanged build outputs across your team.
|
||||||
|
|
||||||
|
Remote Build Cache Server (RBCS) allows teams to share and reuse unchanged build and test outputs,
|
||||||
|
significantly reducing build times for both local and CI environments. By eliminating redundant work,
|
||||||
|
RBCS helps teams become more productive and efficient.
|
||||||
|
|
||||||
|
**Key Features:**
|
||||||
|
- Support for both Gradle and Maven build environments
|
||||||
|
- Pluggable storage backends (in-memory, disk-backed, memcached)
|
||||||
|
- Flexible authentication (HTTP basic or TLS certificate)
|
||||||
|
- Role-based access control
|
||||||
|
- Request throttling
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
- [Quickstart](#quickstart)
|
||||||
|
- [Integration with build tools](#integration-with-build-tools)
|
||||||
|
- [Use RBCS with Gradle](#use-rbcs-with-gradle)
|
||||||
|
- [Use RBCS with Maven](#use-rbcs-with-maven)
|
||||||
|
- [Server configuration](#server-configuration)
|
||||||
|
- [Authentication](#authentication)
|
||||||
|
- [HTTP Basic authentication](#configure-http-basic-authentication)
|
||||||
|
- [TLS client certificate authentication](#configure-tls-certificate-authentication)
|
||||||
|
- [Authentication & Access Control](#access-control)
|
||||||
|
- [Plugins](#plugins)
|
||||||
|
- [Client Tools](#rbcs-client)
|
||||||
|
- [Logging](#logging)
|
||||||
|
- [FAQ](#faq)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Remote Build Cache Server (shortened to RBCS) allows you to share and reuse unchanged build
|
Remote Build Cache Server (shortened to RBCS) allows you to share and reuse unchanged build
|
||||||
and test outputs across the team. This speeds up local and CI builds since cycles are not wasted
|
and test outputs across the team. This speeds up local and CI builds since cycles are not wasted
|
||||||
re-building components that are unaffected by new code changes. RBCS supports both Gradle and
|
re-building components that are unaffected by new code changes. RBCS supports both Gradle and
|
||||||
@@ -12,7 +52,7 @@ and throttling.
|
|||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
### Downloading the jar file
|
### Use the all-in-one jar file
|
||||||
You can download the latest version from [this link](https://gitea.woggioni.net/woggioni/-/packages/maven/net.woggioni:rbcs-cli/)
|
You can download the latest version from [this link](https://gitea.woggioni.net/woggioni/-/packages/maven/net.woggioni:rbcs-cli/)
|
||||||
|
|
||||||
|
|
||||||
@@ -25,7 +65,7 @@ java -jar rbcs-cli.jar server
|
|||||||
By default it will start an HTTP server bound to localhost and listening on port 8080 with no authentication,
|
By default it will start an HTTP server bound to localhost and listening on port 8080 with no authentication,
|
||||||
writing data to the disk, that you can use for testing
|
writing data to the disk, that you can use for testing
|
||||||
|
|
||||||
### Using the Docker image
|
### Use the Docker image
|
||||||
You can pull the latest Docker image with
|
You can pull the latest Docker image with
|
||||||
```bash
|
```bash
|
||||||
docker pull gitea.woggioni.net/woggioni/rbcs:latest
|
docker pull gitea.woggioni.net/woggioni/rbcs:latest
|
||||||
@@ -34,41 +74,16 @@ docker pull gitea.woggioni.net/woggioni/rbcs:latest
|
|||||||
By default it will start an HTTP server bound to localhost and listening on port 8080 with no authentication,
|
By default it will start an HTTP server bound to localhost and listening on port 8080 with no authentication,
|
||||||
writing data to the disk, that you can use for testing
|
writing data to the disk, that you can use for testing
|
||||||
|
|
||||||
### Using the native executable
|
### Use the native executable
|
||||||
If you are on a Linux X86_64 machine you can download the native executable
|
If you are on a Linux X86_64 machine you can download the native executable
|
||||||
from [here](https://gitea.woggioni.net/woggioni/-/packages/maven/net.woggioni:rbcs-cli/).
|
from [here](https://gitea.woggioni.net/woggioni/-/packages/maven/net.woggioni:rbcs-cli/).
|
||||||
It behaves the same as the jar file but it doesn't require a JVM and it has faster startup times.
|
It behaves the same as the jar file but it doesn't require a JVM and it has faster startup times.
|
||||||
becausue of GraalVm's [closed-world assumption](https://www.graalvm.org/latest/reference-manual/native-image/basics/#static-analysis),
|
becausue of GraalVm's [closed-world assumption](https://www.graalvm.org/latest/reference-manual/native-image/basics/#static-analysis),
|
||||||
the native executable does not supports plugins, so it comes with all plugins embedded into it.
|
the native executable does not supports plugins, so it comes with all plugins embedded into it.
|
||||||
|
|
||||||
## Usage
|
## Integration with build tools
|
||||||
|
|
||||||
### Configuration
|
### Use RBCS with Gradle
|
||||||
The location of the `rbcs-server.xml` configuration file depends on the operating system,
|
|
||||||
Alternatively it can be changed setting the `RBCS_CONFIGURATION_DIR` environmental variable or `net.woggioni.rbcs.conf.dir` Java system property
|
|
||||||
to the directory that contain the `rbcs-server.xml` file.
|
|
||||||
|
|
||||||
The server configuration file follows the XML format and uses XML schema for validation
|
|
||||||
(you can find the schema for the main configuration file [here](https://gitea.woggioni.net/woggioni/rbcs/src/branch/master/rbcs-server/src/main/resources/net/woggioni/rbcs/server/schema/rbcs-server.xsd)).
|
|
||||||
|
|
||||||
The configuration values are enclosed inside XML attribute and support system property / environmental variable interpolation.
|
|
||||||
As an example, you can configure RBCS to read the server port number from the `RBCS_SERVER_PORT` environmental variable
|
|
||||||
and the bind address from the `rbc.bind.address` JVM system property with
|
|
||||||
|
|
||||||
```xml
|
|
||||||
<bind host="${sys:rpc.bind.address}" port="${env:RBCS_SERVER_PORT}"/>
|
|
||||||
```
|
|
||||||
|
|
||||||
Full documentation for all tags and attributes is available [here](doc/server_configuration.md).
|
|
||||||
|
|
||||||
### Plugins
|
|
||||||
If you want to use memcache as a storage backend you'll also need to download [the memcache plugin](https://gitea.woggioni.net/woggioni/-/packages/maven/net.woggioni:rbcs-server-memcache/)
|
|
||||||
|
|
||||||
Plugins need to be stored in a folder named `plugins` in the located server's working directory
|
|
||||||
(the directory where the server process is started). They are shipped as TAR archives, so you need to extract
|
|
||||||
the content of the archive into the `plugins` directory for the server to pick them up.
|
|
||||||
|
|
||||||
### Using RBCS with Gradle
|
|
||||||
|
|
||||||
Add this to the `settings.gradle` file of your project
|
Add this to the `settings.gradle` file of your project
|
||||||
|
|
||||||
@@ -113,7 +128,7 @@ add `org.gradle.caching=true` to your `<project>/gradle.properties` or run gradl
|
|||||||
|
|
||||||
Read [Gradle documentation](https://docs.gradle.org/current/userguide/build_cache.html) for more detailed information.
|
Read [Gradle documentation](https://docs.gradle.org/current/userguide/build_cache.html) for more detailed information.
|
||||||
|
|
||||||
### Using RBCS with Maven
|
### Use RBCS with Maven
|
||||||
|
|
||||||
1. Create an `extensions.xml` in `<project>/.mvn/extensions.xml` with the following content
|
1. Create an `extensions.xml` in `<project>/.mvn/extensions.xml` with the following content
|
||||||
```xml
|
```xml
|
||||||
@@ -143,6 +158,46 @@ Alternatively you can set those properties in your `<project>/pom.xml`
|
|||||||
Read [here](https://maven.apache.org/extensions/maven-build-cache-extension/remote-cache.html)
|
Read [here](https://maven.apache.org/extensions/maven-build-cache-extension/remote-cache.html)
|
||||||
for more informations
|
for more informations
|
||||||
|
|
||||||
|
|
||||||
|
## Server configuration
|
||||||
|
RBCS reads an XML configuration file, by default named `rbcs-server.xml`.
|
||||||
|
The expected location of the `rbcs-server.xml` file depends on the operating system,
|
||||||
|
if the configuration file is not found a default one will be created and its location is printed
|
||||||
|
on the console
|
||||||
|
|
||||||
|
```bash
|
||||||
|
user@76a90cbcd75d:~$ rbcs-cli server
|
||||||
|
2025-01-01 00:00:00,000 [INFO ] (main) n.w.r.c.impl.commands.ServerCommand -- Creating default configuration file at '/home/user/.config/rbcs/rbcs-server.xml'
|
||||||
|
```
|
||||||
|
|
||||||
|
Alternatively it can be changed setting the `RBCS_CONFIGURATION_DIR` environmental variable or `net.woggioni.rbcs.conf.dir`
|
||||||
|
Java system property to the directory that contain the `rbcs-server.xml` file.
|
||||||
|
It can also be directly specified from the command line with
|
||||||
|
```bash
|
||||||
|
java -jar rbcs-cli.jar server -c /path/to/rbcs-server.xml
|
||||||
|
```
|
||||||
|
|
||||||
|
The server configuration file follows the XML format and uses XML schema for validation
|
||||||
|
(you can find the schema for the `rbcs-server.xml` configuration file [here](https://gitea.woggioni.net/woggioni/rbcs/src/branch/master/rbcs-server/src/main/resources/net/woggioni/rbcs/server/schema/rbcs-server.xsd)).
|
||||||
|
|
||||||
|
The configuration values are enclosed inside XML attribute and support system property / environmental variable interpolation.
|
||||||
|
As an example, you can configure RBCS to read the server port number from the `RBCS_SERVER_PORT` environmental variable
|
||||||
|
and the bind address from the `rbc.bind.address` JVM system property with
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<bind host="${sys:rpc.bind.address}" port="${env:RBCS_SERVER_PORT}"/>
|
||||||
|
```
|
||||||
|
|
||||||
|
Full documentation for all tags and attributes and configuration file examples
|
||||||
|
are available [here](doc/server_configuration.md).
|
||||||
|
|
||||||
|
### Plugins
|
||||||
|
If you want to use memcache as a storage backend you'll also need to download [the memcache plugin](https://gitea.woggioni.net/woggioni/-/packages/maven/net.woggioni:rbcs-server-memcache/)
|
||||||
|
|
||||||
|
Plugins need to be stored in a folder named `plugins` in the located server's working directory
|
||||||
|
(the directory where the server process is started). They are shipped as TAR archives, so you need to extract
|
||||||
|
the content of the archive into the `plugins` directory for the server to pick them up.
|
||||||
|
|
||||||
## Authentication
|
## Authentication
|
||||||
|
|
||||||
RBCS supports 2 authentication mechanisms:
|
RBCS supports 2 authentication mechanisms:
|
||||||
@@ -250,7 +305,11 @@ as a health check (mind you need to have `Healthcheck` role in order to perform
|
|||||||
|
|
||||||
RBCS ships with a command line client that can be used for testing, benchmarking or to manually
|
RBCS ships with a command line client that can be used for testing, benchmarking or to manually
|
||||||
upload/download files to the cache. It must be configured with the `rbcs-client.xml`,
|
upload/download files to the cache. It must be configured with the `rbcs-client.xml`,
|
||||||
whose location follows the same logic of the `rbcs-server.xml`
|
whose location follows the same logic of the `rbcs-server.xml`.
|
||||||
|
The `rbcs-client.xml` must adhere to the [rbcs-client.xsd](rbcs-client/src/main/resources/net/woggioni/rbcs/client/schema/rbcs-client.xsd)
|
||||||
|
XML schema
|
||||||
|
|
||||||
|
The documentation for the `rbcs-client.xml` configuration file is available [here](conf/client_configuration.md)
|
||||||
|
|
||||||
### GET command
|
### GET command
|
||||||
|
|
||||||
@@ -263,6 +322,24 @@ java -jar rbcs-cli.jar client -p $CLIENT_PROFILE_NAME get -k $CACHE_KEY -v $FILE
|
|||||||
```bash
|
```bash
|
||||||
java -jar rbcs-cli.jar client -p $CLIENT_PROFILE_NAME put -k $CACHE_KEY -v $FILE_TO_BE_UPLOADED
|
java -jar rbcs-cli.jar client -p $CLIENT_PROFILE_NAME put -k $CACHE_KEY -v $FILE_TO_BE_UPLOADED
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you don't specify the key, a UUID key based on the file content will be used,
|
||||||
|
if you add the `-i` command line parameter, the uploaded file will be served with
|
||||||
|
`Content-Disposition: inline` HTTP header so that browser will attempt to render
|
||||||
|
it in the page instead of triggering a file download (in this way you can create a temporary web page).
|
||||||
|
|
||||||
|
The client will try to detect the file mime type upon upload but if you want to be sure you can specify
|
||||||
|
it manually with the `-t` parameter.
|
||||||
|
|
||||||
|
### Benchmark command
|
||||||
|
|
||||||
|
```bash
|
||||||
|
java -jar rbcs-cli.jar client -p $CLIENT_PROFILE_NAME benchamrk -s 4096 -e 10000
|
||||||
|
```
|
||||||
|
This will insert 10000 randomly generates entries of 4096 bytes into RBCS, then retrieve them
|
||||||
|
and check that the retrieved value matches what was inserted.
|
||||||
|
It will also print throughput stats on the way.
|
||||||
|
|
||||||
## Logging
|
## Logging
|
||||||
|
|
||||||
RBCS uses [logback](https://logback.qos.ch/) and ships with a [default logging configuration](./conf/logback.xml) that
|
RBCS uses [logback](https://logback.qos.ch/) and ships with a [default logging configuration](./conf/logback.xml) that
|
||||||
|
@@ -137,9 +137,22 @@ Configures TLS encryption.
|
|||||||
read-timeout="PT5S"
|
read-timeout="PT5S"
|
||||||
write-timeout="PT5S"/>
|
write-timeout="PT5S"/>
|
||||||
<event-executor use-virtual-threads="true"/>
|
<event-executor use-virtual-threads="true"/>
|
||||||
|
|
||||||
<cache xs:type="rbcs:inMemoryCacheType" max-age="P7D" enable-compression="false" max-size="0x10000000" />
|
<cache xs:type="rbcs:inMemoryCacheType" max-age="P7D" enable-compression="false" max-size="0x10000000" />
|
||||||
<!--cache xs:type="rbcs:fileSystemCacheType" max-age="P7D" enable-compression="false" path="${sys:java.io.tmpdir}/rbcs"/-->
|
|
||||||
<authorization>
|
<!-- uncomment this to enable the filesystem storage backend, sotring cache data in "${sys:java.io.tmpdir}/rbcs"
|
||||||
|
<cache xs:type="rbcs:fileSystemCacheType" max-age="P7D" enable-compression="false" path="${sys:java.io.tmpdir}/rbcs"/>
|
||||||
|
-->
|
||||||
|
|
||||||
|
<!-- uncomment this to use memcache as the storage backend, also make sure you have
|
||||||
|
the memcache plugin installed in the `plugins` directory if you are using running
|
||||||
|
the jar version of RBCS
|
||||||
|
<cache xs:type="rbcs-memcache:memcacheCacheType" max-age="P7D" chunk-size="0x1000" digest="MD5">
|
||||||
|
<server host="127.0.0.1" port="11211" max-connections="256"/>
|
||||||
|
</cache>
|
||||||
|
-->
|
||||||
|
|
||||||
|
<authorization>
|
||||||
<users>
|
<users>
|
||||||
<user name="user1" password="II+qeNLft2pZ/JVNo9F7jpjM/BqEcfsJW27NZ6dPVs8tAwHbxrJppKYsbL7J/SMl">
|
<user name="user1" password="II+qeNLft2pZ/JVNo9F7jpjM/BqEcfsJW27NZ6dPVs8tAwHbxrJppKYsbL7J/SMl">
|
||||||
<quota calls="100" period="PT1S"/>
|
<quota calls="100" period="PT1S"/>
|
||||||
|
Reference in New Issue
Block a user