seaweedFS

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.

support map reduce in PureGo, Lua, UnixPipes

seaweedfs kubernetes operator

SeaweedFS CSI Driver

https://github.com/chrislusf/seaweedfs.git

白皮书： https://github.com/chrislusf/seaweedfs/wiki/SeaweedFS_Architecture.pdf

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

https://github.com/chrislusf/seaweedfs.wiki.git