Micro-service for file storage and processing written in Go
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

322 lines
4.9 KiB

2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
  1. # Lithium
  2. Micro-service for file storage and processing written in Go.
  3. ## Features
  4. - Image processing pipelines for various transformations
  5. - Web API for storing & retrieving files
  6. ## Requirements
  7. - [Go 1.17+](https://go.dev/)
  8. - [_Docker_](https://docs.docker.com/) (optional)
  9. ## Setup
  10. #### 1. Clone repository
  11. ```shell
  12. git clone git@gogs.informatik.hs-fulda.de:FabianVowie/Lithium.git
  13. ```
  14. #### 2. Pull dependencies
  15. ```shell
  16. go get
  17. ```
  18. #### 3. Build & start application
  19. ```shell
  20. go run .
  21. ```
  22. **Run using [Docker](https://docs.docker.com/) container**
  23. ```shell
  24. docker run --rm -p 8000:8000 -v "$PWD":/usr/src/lithium -w /usr/src/lithium golang:1.17 go run .
  25. ```
  26. ## Testing
  27. ```shell
  28. go test ./...
  29. ```
  30. ### Run tests in verbose logging mode
  31. ```shell
  32. go test ./... -v
  33. ```
  34. ### Run tests in [Docker](https://docs.docker.com/) container
  35. ```shell
  36. docker run --rm -v "$PWD":/usr/src/lithium -w /usr/src/lithium golang:1.17 go test ./...
  37. ```
  38. ## Configuration
  39. Config options can be adjusted via the [`settings.json`](settings.json) file in the root directory.
  40. ```json
  41. {
  42. "endpoint": "0.0.0.0:8000",
  43. "token": "changeme",
  44. "rate_limiter": {
  45. "requests_per_minute": 20,
  46. "allowed_burst": 5
  47. },
  48. "storage_provider": {
  49. "type": 0,
  50. "base_path": "assets"
  51. }
  52. }
  53. ```
  54. ## Rate Limiting
  55. By default, the rate limiting takes place on a per-route basis. When the limit for a specific route is hit, the response will return a status code `429: Too Many Requests`.
  56. | Headers | Explanation |
  57. | --------------------- | -------------------------------- |
  58. | X-Ratelimit-Limit | Allowed requests per minute |
  59. | X-Ratelimit-Remaining | Remaining requests |
  60. | X-Ratelimit-Reset | Seconds until requests replenish |
  61. ## API
  62. ### `GET` `/`
  63. Show application information.
  64. **Required headers**:
  65. ```shell
  66. Authorization: Bearer <Token>
  67. ```
  68. **Example response**:
  69. ```json
  70. {
  71. "name": "Lithium",
  72. "version": "0.1.0"
  73. }
  74. ```
  75. ### `GET` `/files`
  76. Show files.
  77. **Required headers**:
  78. ```shell
  79. Authorization: Bearer <Token>
  80. ```
  81. **Example response**:
  82. ```json
  83. {
  84. "files": [
  85. {
  86. "bucket": "foo",
  87. "file_name": "output.jpg",
  88. "size": 14523,
  89. "modified_at": "2022-02-14T14:19:07.491308411+01:00"
  90. }
  91. ]
  92. }
  93. ```
  94. ### `GET` `/pipelines/{pipeline}`
  95. Show pipeline information.
  96. **Required headers**:
  97. ```shell
  98. Authorization: Bearer <Token>
  99. ```
  100. **Example response**:
  101. ```json
  102. {
  103. "name": "example pipeline",
  104. "slug": "example",
  105. "type": 0,
  106. "remove_metadata": false,
  107. "steps": [
  108. {
  109. "name": "resize image",
  110. "type": 0,
  111. "options": {
  112. "width": 1280,
  113. "height": 720,
  114. "upscale": false
  115. }
  116. }
  117. ],
  118. "output": {
  119. "format": "jpg",
  120. "quality": 90
  121. }
  122. }
  123. ```
  124. ## Pipelines
  125. The project uses a pipeline system defined by [JSON](https://en.wikipedia.org/wiki/JSON) files located in the [config](config) folder.
  126. Take a look at the [example pipeline configuration file](config/example.json).
  127. Available pipeline `type`s: `0` (Image), `1` (Video)
  128. ```json
  129. {
  130. "name": "example pipeline",
  131. "slug": "example",
  132. "type": 0,
  133. "removeMetadata": false,
  134. "steps": [],
  135. "output": {}
  136. }
  137. ```
  138. ### Images pipeline
  139. The image pipeline offers the following additional output options.
  140. Available `format` types: `jpg` (or `jpeg`), `png`, `gif`, `tif` (or `tiff`) and `bmp`
  141. The `quality` field can contain any integer between 1 and 100.
  142. ```json
  143. {
  144. "output": {
  145. "format": "jpg",
  146. "quality": 90
  147. }
  148. }
  149. ```
  150. ### Available pipeline steps
  151. Each pipeline step consists of an optional `name`, a predefined `type` and configurable `options`.
  152. See the available options below for more information.
  153. ```json
  154. {
  155. "name": "step name",
  156. "type": 0,
  157. "options": {}
  158. }
  159. ```
  160. #### Resizing images
  161. Resize an image by a given `width` and `height`.
  162. **Step definition**:
  163. ```json
  164. {
  165. "type": 0,
  166. "options": {
  167. "width": 1280,
  168. "height": 720
  169. }
  170. }
  171. ```
  172. #### Rotating images
  173. Rotate an image with a given `angle` in degrees.
  174. **Step definition**:
  175. ```json
  176. {
  177. "type": 1,
  178. "options": {
  179. "angle": 90.0
  180. }
  181. }
  182. ```
  183. #### Flipping images
  184. Flip an image with a given direction.
  185. Allowed values for the `direction` option are `"h"` (horizontal), `"v"` (vertical)
  186. **Step definition**:
  187. ```json
  188. {
  189. "type": 2,
  190. "options": {
  191. "direction": "h"
  192. }
  193. }
  194. ```
  195. #### Grayscale
  196. Convert the colorspace of an image into grayscale.
  197. **Step definition**:
  198. ```json
  199. {
  200. "type": 3
  201. }
  202. ```
  203. #### Fit
  204. Scales down the image to fit the specified maximum width and height.
  205. **Step definition**:
  206. ```json
  207. {
  208. "type": 4,
  209. "options": {
  210. "height": 300,
  211. "width": 200
  212. }
  213. }
  214. ```
  215. #### Invert
  216. Invert image colors.
  217. **Step definition**:
  218. ```json
  219. {
  220. "type": 5
  221. }
  222. ```
  223. #### Blur
  224. Blur image using Gaussian functions.
  225. **Step definition**:
  226. ```json
  227. {
  228. "type": 6,
  229. "options": {
  230. "sigma": 50.0
  231. }
  232. }
  233. ```
  234. ## Authors
  235. - [Fabian Vowie](https://gogs.informatik.hs-fulda.de/FabianVowie)
  236. - [Roman Zipp](https://gogs.informatik.hs-fulda.de/roman.zipp)