README.md

# fs

The package `fs` offers a more complete altenative to the `io/fs` package of the Go standard library.

The idea was to find common abstractions for paths and filesystems, no matter, if they are
local, remote, backed by memory, drives of cloud servers.

## API docs

see https://pkg.go.dev/gitlab.com/golang-utils/fs

## design decisions

If became clear soon, that such a general `fs` library can't be done without rethinking and integrating
the concept of a `path`.

Paths should work in a similar way, no matter, if they are local windows paths, windows UNC paths or unix
paths. No matter if the path is part of a URL or local, if it has a drive letter or a network share.

The `fs/path` package makes Path a proper type and has *relative* and *absolute* paths as _different types_.
Every path can have a relative part and every path can be represented by a `string`.

Therefor the common `path.Path` interface is as follows:

```go
type Path interface {
	String() string
	Relative() Relative
}
```

where `path.Relative` is just a `string`:

```go
type Relative string
```

However, since a `path.Relative` is also a `path.Path`, it implements that interface:

```go
func (r Relative) String() string {
	return string(r)
}

func (r Relative) Relative() Relative {
	return r
}
```

Very simple. It became clear, that an absolute path is always related to a filesystem, while a relative path
is independant of the filesystem. Therefor `path.Absolute` became an interface, since the differences
between local os paths and also remote paths show up in their absolute part:

```go
type Absolute interface {
	Path
	Head() string
}
```

So a `path.Absolute` is just a `path.Path` with a `Head`.
The easiest implementation of this `path.Absolute` is a local path:

```go
type Local [2]string

func (a Local) Head() string {
	return a[0]
}

func (a Local) String() string {
	return a[0] + a[1]
}

func (a Local) Relative() Relative {
	return Relative(a[1])
}
```

Here some design decision come into place:

1. the relative part of a path always uses the slash `/` as a separator.
2. directories always end with a `/`. this makes it easy to check, if a path refering to a directory without the
   the need of a filesystem.
3. the head of an absolute path is always a directory and therefor it always ends with a slash `/`
4. parts of paths are joined together simply by glueing them together. Since a directory must end in a slash `/` this
   naturally leads to correct paths. 
5. the head of an absolute path is depending on the filesystem that it refers to: e.g.
   - a local windows paths starts with a drive letter, e.g. `c:/`
   - a windows share in UNC starts with the host followed by the share name, e.g. `\\example.com/share/`
   - a url starts with a schema, followed by a host `http://example.com/`
6. absolute paths can be written differently, e.g.
   - `c:/` can also be written as `C:\`
   - `\\example.com/share/` can also be written as `\\example.com\share\`
   therefor we need one unique internal representation, while allowing the path to be generated by parsing also
   the alternative ways of writing. this leads to parsers for converting an absolute path string to a `path.Absolute`.
7. While having a unified syntax behind the scenes, `path.Local` can be converted to the most typical string notation
   that is used on the running system, by calling the `ToSystem` method, so that it can easily be integrated with external tools
8. The only time where a local absolute path is being created via solo a relative path, is when the relative path is relative
   to the current working directory of the system. This case is handled like every other way to convert a string to
	 a `path.Local` by the `path.ParseLocal()` function (see below) 

## Local and Remote

Not only because of the different ways the local and remote absolute paths are written, but also because of the 
very different performance characteristics and optimization opportunities, it makes sense to be able to
distinguish between *local* and *remote* paths, while still being able to handle them both as *absolute* paths.

This is taken into account via having `path.Local` as well as `path.Remote` implement the `path.Absolute` interface.
This way we end up with two parsers: 
- `ParseLocal(string)` handling local windows, UNC and unix paths, also paths relativ to the working directory, e.g. `./a/`
- `ParseRemote(string)` handling URLs

A `path.Remote` is basically just a wrapper around an `url.URL` that is implementing the `path.Absolute` interface.

```go
type Remote struct {
	*url.URL
}

func (u *Remote) Relative() Relative {...}
func (u *Remote) Head() string {...}

```

There are some helpers, to get the common string notation for windows, UNC and so on back, so that everything does integrate well.

## filesystems

Since it became clear that *absolute* paths are associated with a filesystem, this lead to filesystems being initiated
via an `path.Absolute`.

For good integration with the existing `io/fs.FS` interface which provides only a solution for reading access, we started with the `fs.ReadOnly` filesystem interface:

```go
type ReadOnly interface {
	Reader(p path.Relative) (io.ReadCloser, error)
	Exists(p path.Relative) bool
	ModTime(p path.Relative) (time.Time, error)
	Abs(p path.Relative) path.Absolute
	Size(p path.Relative) int64
}
```

It is very easy to convert an existing `io/fs.FS` implementation to the `fs.ReadOnly` interface via the `wrapfs` package:

```go
dir := "/etc"
fsys := os.DirFS(dir)
ro, err := wrapfs.New(fsys, path.MustLocal(dir+"/")) 
size := ro.Size("fstab")
```

For `os.DirFS` there is an easier way via the `localfs` package:

```go
fs, err := localfs.New(path.MustLocal("/etc/"))
size := fs.Size("fstab")
```

The real power comes with the more general `fs.FS` interface:

```go
type FS interface {
	ReadOnly
	ExtWriteable
	ExtDeleteable
}
```

It adds to the `ReadOnly` interface the ability to write and delete files and folders. This is also implemented by the `localfs` package:

```go
fs, err := localfs.New(path.MustLocal(`C:\`))
recursive := true
err = fs.Delete(path.Relative("Windows/"), recursive)
```

But the same powerful interface is also implemented by the `httpsfs` package which accesses a filesystem via `http`:

```go
fs, err := httpsfs.New(path.MustRemote(`http://localhost:3030/data/`))
createDirs := true
err = fs.Write(path.Relative("myblog/january/something-new.txt"), fs.ReadCloser(strings.NewReader("some text")), createDirs)
```

Finally we have some properties specifically for local filesystems that we don't have for remote filesystems and vice versa:

```go
fs, err := localfs.New(path.MustLocal(`/`))
err = fs.SetMode(path.Relative("etc/"), 0750)
```

```go
fs, err := httpsfs.New(path.MustRemote(`http://localhost:3030/data/`))
meta := map[string][]byte{"Content-Type": []byte("application/json")}
data := fs.ReadCloser(strings.NewReader(`{key: "val"}`))
err = fs.WriteWithMeta(path.Relative("sth.json"), data, meta, true) 
```

So we have this hierarchy of FS interfaces where the last ones a more specific but also more powerfull and the
first ones are more general and easier to implement:

```go
type ReadOnly interface {
	Reader(p path.Relative) (io.ReadCloser, error)
	Exists(p path.Relative) bool
	ModTime(p path.Relative) (time.Time, error)
	Abs(p path.Relative) path.Absolute
	Size(p path.Relative) int64 
}

type FS interface {
	ReadOnly
	ExtWriteable
	ExtDeleteable
}

type Local interface {
	FS
	ExtMoveable
	ExtModeable
	ExtRenameable
	ExtSpaceReporter
}

type Remote interface {
	FS
	ExtMeta
	ExtURL
}
```

Finally we have `TestFS` interface that is covering everything, so that can easily test our packages against all features:

```go
type TestFS interface {
	Local
	Remote
}
```

The `mockfs` package offers an implementation of the `fs.TestFS` interface that is backed by a map for easy testing.

## For implementors

For implementors there is a large test suite that can be easily integrated into your package testing to ensure that
your filesystem behaves correctly according to the specifications. Here an example how to use it, based on the `mockfs` package:

```go
package mockfs

import (
	"testing"

	"gitlab.com/golang-utils/fs"
	"gitlab.com/golang-utils/fs/path"
	"gitlab.com/golang-utils/fs/spec"
)

func mustNew(loc path.Absolute) fs.TestFS {
	f, err := New(loc)
	if err != nil {
		panic(err.Error())
	}
	return f
}

func TestSpec(t *testing.T) {
	var c spec.Config
	s := spec.TestFS(c, mustNew)
	s.Run("", t)
}
```