Merge branch 'implement-search-9' into 'master'

## Overview

> TODO update this diagram

> TODO: update this diagram

![Architecture diagram](assets/architecture-diagram.jpg 'Diagram of Minds architecture')

[Click to enlarge](assets/architecture-diagram.jpg)

## Structure

The backend is split into two main areas, **Core** and **Controllers**.

Minds follows an abstraction design pattern - by separating complex processes into standalone chunks with singular responsibilities, it's easier to conduct tests and update infrastructure.

**Core** is where all the heavy lifting modules and internal apis exist.

The backend is split into two main areas, **Core** and **Controllers**:

**Controllers** support both _CLI_ and _REST_ api's. These are publicly exposed, as are the public interfaces for the **Core** modules and managers.

- **Core** is where all the heavy lifting [modules](../walk-throughs/backend-modules) and internal apis exist.

- **Controllers** support both _CLI_ and _REST_ api's. These are publicly exposed, as are the public interfaces for the **Core** modules and managers.

```

engine

└───...

```

### Abstraction design

Minds follows an abstraction design pattern - by separating complex processes into standalone chunks with singular responsibilities, it's easier to conduct tests and update infrastructure.

## Modules

To illustrate how modules work, we're going to create a VideoChat module that makes use of dependency injection providers.

Our completed module will look something like this:

![Module diagram](assets/engine-module-diagram.png 'Diagram of an example VideoChat module')

[Click to enlarge](assets/engine-module-diagram.png)

### Module building blocks

#### The Model

`Core/VideoChat/VideoChat.php`

- The characteristics that define a single unit of whatever content your module is going to handle (e.g. a single video chat might include an array of _participantGuids, startTimestamp, endTimestamp_, etc.

- You'll want to `use Minds\Traits\MagicAttributes` so you can automatically use _get, set, and is_ methods on each of your model's properties

- If you'll be using an api endpoint to interact with the frontend, include a `public function export() {...}` here so it can be picked up by your controller's `Factory::exportable()` function, which transforms arrays into an exportable format

- May include functions for calculated fields (e.g. `getUrn()`)

#### The Manager

`Core/VideoChat/Manager.php`

- Interfaces with the repository

- Hydrate the entities returned in the response here (if needed)

- Functions might include `get()`, `getList()`, `add()`, `update()`, `delete()`

- A given function might query the repository, hydrate entity guids that are returned in the response, and call delegates

```php

<?php

namespace Minds\Core\VideoChat;

class Manager

{

    /** @var Repository $repository */

    private $repository;

    /** @var NotificationDelegate $notificationDelegate */

    private $notificationDelegate;

    public function __construct($repository = null, $notificationDelegate = null)

    {

        $this->repository = $repository ?: Di::_('Repository');

        $this->notificationDelegate = $notificationDelegate ?: new Delegates/NotificationDelegate;

    }

    /**

     * Add a model

     * @param Model $model

     * @return void

     */

    public function add(Model $model)

    {

        $this->repository->add($model);

        $this->notificationDelegate->onAdd($model);

    }

```

_Note how we are able to switch out repositories here without impacting the wider codebase by interacting with our Manager. During migrations, a double writing pattern could be added with a few lines._

#### The Repository

`Core/VideoChat/Repository.php`

- Interfaces with the database.

- Should **only ever** be referenced by its _Manager_. This single point of contact is essential because it makes it easy to replace legacy databases with new ones that are more suitable for our needs

- Should `use Minds\Common\Repository\Response`

- Functions might include `get()`, `getList()`, `add()`, `update()`, `delete()`

```php

<?php

class Repository

{

    /** @var Client $db */

    private $db

    public function __construct($db = null)

    {

        $this->db = $db ?: Di::_()->get('Database\Cassandra\Cql');

    }

    public function add(Model $model)

    {

        ...

    }

...

```

#### The Provider

`Core/VideoChat/Provider.php`

- Defines a function that registers your _Manager_ to make it available for dependency injection

```php

<?php

namespace Minds\Core\VideoChat;

use Minds\Core\Di\Provider as DiProvider;

class Provider extends DiProvider

{

    public function register()

    {

        $this->di->bind('VideoChat\Manager', function ($di) {

            return new Manager();

        }, ['useFactory' => true]);

    }

}

```

#### The Module

`Core/VideoChat/Module.php`

- Creates an instance of your _Provider_ and calls its `register()` function.

```php

<?php

namespace Minds\Core\VideoChat;

use Minds\Interfaces\ModuleInterface;

class Module implements ModuleInterface

{

    public function onInit()

    {

        $provider = new Provider();

        $provider->register();

    }

}

```

- Modules must be registered in [Core/Minds.php](https://gitlab.com/minds/engine/blob/master/Core/Minds.php):

```php

private $modules = [

   ...

    VideoChat\Module::class,

];

```

#### Delegates

`Core/VideoChat/Delegates/NotificationDelegate.php`

_You can give your delegate any name you wish, we are using NotificationDelegate as an example_

- Use delegates to keep the _Manager_ clean. Delegates should be used to execute small, stateless outbound functions that don't return values that need further processing. (If this doesn't happen in your module, you don't need to use delegates)

```php

<?php

class NotificationDelegate

{

    public function onAdd(Model $model)

    {

        ...

    }

...

```

#### Testing your module

Managers, Repositories and Delegates should have 100% [spec test](#spec-tests) coverage.

> See the [backend modules walk-through](../walk-throughs/backend-modules) for detailed information about how to build a core module

## Events

> TODO: Add examples of how each type is used

There are two types of events handling in the backend:

- **Internal events dispatcher** - when you want to do something inside the same thread

- **Event queue** - go to the background and do something at another time

- **Event queue** - when it's ok for the process to go to the background and do something at another time

## Runners

Minds uses [phpspec](https://www.phpspec.net/en/stable/) and encourages test-first development.

Specs are highly abstracted alter-egos of your _Manager_, _Repository_ and _Delegate_ files that allow you to focus on the bare-bones concepts of what those files will eventually do when they reach their final form (so you can plan effectively before tackling practical technicalities and specifics).

Make sure you include `@package Minds\Core\<<VideoChat>>` in `<<VideoChat>>.php` so it can be picked up by phpspec in `<<VideoChatSpec>>.php`.

### Executing

To run all tests:

```console

bin/phpspec run

```

To run a specific spec file (or folder), add its name:

```console

bin/phpspec run Spec/Core/VideoChats/Repository.php

```

To run a specific test inside of that spec file, add its starting line number:

```console

  bin/phpspec run Spec/Core/VideoChats/Repository.php:42

```

### Verbose output

Running this command will give you deep output of the tests:

```console

bin\phpspec run -vvv

```

### Writing new tests

#### Creating a Spec.php file

Run this command to create a test skeleton in the appropriate place with the default classes imported:

```console

bin/phpspec run Minds/Your/Namespace/ClassYouWantToSpec

```

This will create a folder in `minds/engine/`.

#### Mock everything

There's a lot going on under the hood in the modules, some of them have low level connectivity to the data stores. If you don't own the object, mock it. Phpspec Prophecy makes it really easy.

Inside your unit test, be sure to import the class you want to mock.

#### Mocking at the test class level

Phpspec provides a `let()` function that gets run before each test. If you provide a typed parameter, it will be mocked to the matching namespace.

```php

use PhpSpec\ObjectBehavior;

use Minds\Entities\Entity;

class ManagerSpec extends ObjectBehavior {

    protected $entity;

    let(Entity $entity) {

        //Store a reference to the mock for reuse.

        $this->entity = $entity

    }

}

```

#### Mocking at the test function level

Phpspec will provide mocks via function parameters in your test:

```php

use PhpSpec\ObjectBehavior;

use Minds\Entities\Entity;

class ManagerSpec extends ObjectBehavior {

    public function it_should_run_test(Entity $entity) {

    }

}

```

#### Setting expectations

These mockable objects can then configure to simulate what will happen with special functions available on the mock.

`ShouldBeCalled()` will set the expectation that mocked method should be called with a parameter:

```php

$entity->setGuid(123)->shouldBeCalled();

```

`willReturn()` simulates the response of a mocked method:

```php

$entity->setGuid(123)->willReturn($entity);

```

#### Getting to the underlying object

Sometimes, php will choke on the reflection of these mocked objects (especially when constructing other objects).

```php

//This will instantiate an object and still be mockable.

$mockedObject->getList()->willReturn([]);

$service = new ServiceThatNeedsDependencies($mockedObject->getWrappedObject());

```

> See the [backend tests walk-through](../walk-throughs/backend-tests) for detailed info on writing and running tests

## Controllers

run php cli QueueRunner run --runner=NotificationDispatcher

```

Or, for top feeds you could use:

Or, for top feeds at [Controllers/Cli/Top.php](https://gitlab.com/minds/engine/blob/master/Controllers/Cli/Top.php) you could use:

```console

run php cli Top sync_activity

- Most of our data is stored in Cassandra - but we also dual write some things to Elasticsearch because querying/reading in Cassandra can be restrictive and cumbersome.

- All communication with Cassandra should go through a _Repository_ as described in the [module diagram](##Modules).

- See the [infinite-scroll walk-through](../walk-throughs/infinite-scroll.md)

```php

use Minds\Core\Di\Di;

$this->client = $client ?: Di::_()->get('Database\Cassandra\Cql');

```

> See the [infinite-scroll walk-through](../walk-throughs/infinite-scroll.md) for a sample Cassandra interaction

### Elasticsearch

Although we store almost all of our important data in Cassandra, we alse use Elasticsearch because it is better at querying/reading data than Casssandra. Besides metrics, everything in Elasticsearch - inlcuding the newsfeed and top feed = is ephemeral.

Although we store almost all of our important data in Cassandra, we alse use Elasticsearch because it is better at querying/reading data than Casssandra. Besides metrics, everything in Elasticsearch - including the newsfeed and top feed - is ephemeral.

See [Feeds/Repository.php](https://gitlab.com/minds/engine/blob/master/Core/Feeds/Repository.php) for an example that includes dual writing to both Cassandra and Elasticsearch.

## Errors & Exceptions

> Currently being reworked

> TODO: Currently being reworked

## Utilities

#### URNs

Uniform Resource Names (urns) include _type_ information, which adds additional data into GUIDs/UUIDs and helps for indexing. Urns can be packed with multiple parts, each separated by a ":". Because they aren't limited like a single GUID/UUID, they can be used to neatly convey various types of information, including nesting contexts (which is helpful for comment threads - see [Comment.php](https://gitlab.com/minds/engine/blob/master/Core/Comments/Comment.php) for more).

Uniform Resource Names (urns) include _type_ information, which adds additional data into GUIDs/UUIDs and helps for indexing. Urns can be packed with multiple parts, each separated by a colon. Because they aren't limited like a single GUID/UUID, they can be used to neatly convey various types of information, including nesting contexts (which is helpful for comment threads - see [Comment.php](https://gitlab.com/minds/engine/blob/master/Core/Comments/Comment.php) for more).

To create a new urn:

### FPM

```

```console

docker build -t minds/fpm:latest -f containers/php-fpm/Dockerfile .

```

### Runners

```

```console

docker build -t minds/runners:latest -f containers/php-runners/Dockerfile .

```

A kubernetes environment can be created by running:

```

```console

helm upgrade \

    --install \

    --reuse-values \

Do not hard code the values in the configMap, reference them via `.Values.key.subkey`:

```

```console

 // Twillio configuration

    $CONFIG->set('twilio', [

        'account_sid' => '{{ .Values.twilio.sid }}',

- If the answer to a question exists in documentation, share the link to the docs instead of rephrasing the information.

- When you encounter new information not available in GitLab’s documentation (for example, when working on a support case or testing a feature), your first step should be to create a merge request to add this information to the docs. You can then share the MR in order to communicate this information.

- When you encounter new information not available in Minds’s documentation (for example, when working on a support case or testing a feature), your first step should be to create a merge request to add this information to the docs. You can then share or screenshot the MR in order to communicate this information.

### Link instead of summarize

Use [GitHub flavored markdown](https://help.github.com/en/articles/basic-writing-and-formatting-syntax).

#### Headers

### Headers

The largest tier of headers should have _two_ hashes, e.g. `## My h1 primary header`. If you use one hash, it won't work with the navigation sidebar.

- `### My cool subheader with many words`

- `### My cool Minds subheader that contains a proper noun`

## Adding images

- Add the image file to `docs/assets/`

- Add a "Click to enlarge" link below images that depict small details/text

```md

![My cool diagram](assets/my-cool-diagram.png "My cool diagram's alt text")

[Click to enlarge](assets/my-cool-diagram.png)

```

## Editing an existing docs page

Edit by clicking the "edit" button at the top of the docs site page, or by navigating to `docs/` and editing the corresponding document:

Edit by clicking the "edit" button at the top of the docs site page, or by navigating to `docs/` (or `website/blog`, if it's a blog) and editing the corresponding document:

`docs/doc-to-be-edited.md`

Edit me...

```

For more information about docs, click [here](https://docusaurus.io/docs/en/navigation)

## Editing an existing blog post

Edit blog posts by clicking the "edit" button at the top of the docs site page, or by navigating to `website/blog` and editing the corresponding post:

`website/blog/post-to-be-edited.md`

```markdown

---

id: post-needs-edit

title: This Blog Post Needs To Be Edited

---

Edit me...

```

For more information about blog posts, click [here](https://docusaurus.io/docs/en/adding-blog)

Click [here](https://docusaurus.io/docs/en/navigation) for more info on docs, or [here](https://docusaurus.io/docs/en/adding-blog) for blogs.

## Adding Content

// Add newly-created-doc to the Getting Started category of docs

{

  "docs": {

    "Getting Started": [

    "Getting started": [

      "quick-start",

      "newly-created-doc" // new doc here

    ],

}

```

For more information about adding new docs, click [here](https://docusaurus.io/docs/en/navigation)

Re-run `yarn start` to see the changes in the sidebar.

For more information about adding new docs, click [here](https://docusaurus.io/docs/en/navigation).

### Adding a new blog post

    /* you can add custom pages */

    { page: 'help', label: 'Help' },

    /* you can add external links */

    { href: 'https://github.com/facebook/Docusaurus', label: 'GitHub' },

    { href: 'https://gitlab.com/minds', label: 'GitLab' },

    ...

  ],

  ...

}

```

For more information about the navigation bar, click [here](https://docusaurus.io/docs/en/navigation)

For more information about the navigation bar, click [here](https://docusaurus.io/docs/en/navigation).

### Adding custom pages

1. Docusaurus uses React components to build pages. The components are saved as .js files in `website/pages/en`:

1. Docusaurus uses React components to build pages. The components are saved as .js files in `website/pages/en`

1. If you want your page to show up in your navigation header, you will need to update `website/siteConfig.js` to add to the `headerLinks` element:

`website/siteConfig.js`

For more information about custom pages, click [here](https://docusaurus.io/docs/en/custom-pages).

## Full docusaurus documentation

### Adding an image to a docs page

Full documentation can be found on the [docusaurus website](https://docusaurus.io/).

- Add the image file to `docs/assets/`

- Add a "Click to enlarge" link below images that depict small details/text

```md

![My cool diagram](assets/my-cool-diagram.png "My cool diagram's alt text")

[Click to enlarge](assets/my-cool-diagram.png)

```

## Credits

## Resources and credits

Full documentation can be found on the [docusaurus website](https://docusaurus.io/).

A portion of this guide's content is taken from GitLab's excellent [documentation styleguide](https://git.causal.ch/help/development/documentation/styleguide.md).

### Common

In most cases, new code will be stored inside subject-specific module folders. However, if you are making something that will be used throughout the site, put it in `/common/` so it can be easily accessed from other modules. Some examples of the kind of things that belong in `/common/`:

In most cases, new code will be stored inside subject-specific module folders. However, if you are making something that will be used throughout the site, put it in the `common/` folder so it can be easily accessed from other modules. Some examples of the kind of things that belong in `common/`:

- **Directives**: user profile hovercard, tooltips, things related to Material design lite (slider, switch, date time picker), etc.

- **Pipes**: ... examples ...

- What else is in here...

- **Pipes**: TODO

- **Services**: TODO

## Naming conventions

## Spec tests

### Executing

We test our code to prevent software defects and verify that it behaves as we expect it to.

`ng test`

### Cypress tests

> TODO: Brian

#### Select with data attributes, not classes

This is Cypress' best practice for finding testable attributes in the DOM, since using HTML selectors to find positions in the DOM is both brittle and flakey. So wherever you see `data-name-of-component`, you'll know it's e2e related.

For example, to add data attributes to our `minds-activity` objects:

```html

[attr.data-minds-activity-guid]="activity.guid"

```

> See the [frontend tests walk-through](../walk-throughs/frontend-tests) for information on writing and running tests

## Styles

}

```

If something is black or white and you want to _not_ change it when the theme is changed (e.g. an overlay modal background should always be black, regardless of theme), use `$m-black-always` or `$m-white-always`.

If something is black or white and you want it to _not_ change when the theme is changed (e.g. you want an overlay modal background to always be black, regardless of theme), use `$m-black-always` or `$m-white-always`.