Commit cb2bd9b5 authored by David Hendriks's avatar David Hendriks
Browse files

cleaned up BHBH example notebook. working on event based logging notebok

parent cfa009a9
Loading
Loading
Loading
Loading
+103 −5237

File changed.

Preview size limit exceeded, changes collapsed.

+115 −33
Original line number Diff line number Diff line
%% Cell type:markdown id:745f3054-38dc-4fec-9dec-893dbf98a4c8 tags:

# Tutorial: Event based logging output
In this notebook we show how to enable and use the event based logging, which is a set of standardised outputs from `binary_c` for events like supernovae and Roche-lobe overflow episodes.

The events that are available in this version of `binary_c-python` are listed on [the events based logging page in the documentation](https://binary_c.gitlab.io/binary_c-python/event_based_logging_descriptions.html).

The relevant options for this functionality are prepended with `event_based_logging_` (see the [population options documentation](https://binary_c.gitlab.io/binary_c-python/population_options_descriptions.html) and the [binary_c options documentation](https://binary_c.gitlab.io/binary_c-python/binary_c_parameters.html))

The events are all flagged with a uuid that allows for matching events that come from the same system. This is useful for example to track the events that preceded the formation of double compact objects, so we can study their evolutionary path.

%% Cell type:markdown id:ea335f84-992b-492c-a354-10ba78e0f200 tags:

## Enabling event-based logging in `binary_c`
To enabled the event-based logging in binary_c we need pass the `event_based_logging_enable_event_logging=1` setting to `binary_c`. We also need to specify the specific events that we want to have logged, by passing any of the following parameters:

- `event_based_logging_enable_SN_event_logging=1`
- `event_based_logging_enable_RLOF_event_logging=1`

%% Cell type:markdown id:6f3e8475-5381-4c79-944d-c6702cff0d4f tags:

## Enabling event-based log handling in `binary_c-python`
To enable the automatic processing of the event logs with `binary_c-python`, we need to set `event_based_logging_handle_output=1`

This allows `binary_c-python` to automatically process the output of binary_c and create output files that contain the event logs.

There are several options pertaining to the processing of these output files:

 - `event_based_logging_combine_individual_event_files`: whether to automatically combine all the process-specific event log files into one combined file.
 - `event_based_logging_combined_events_filename`: filename of the combined event file.
- `event_based_logging_remove_individual_event_files_after_combining`: whether to remove the process-specific event log files after combining.
- `event_based_logging_split_events_file_to_each_type`: whether to split the combined event file into event-specific files.
- `event_based_logging_remove_original_combined_events_file_after_splitting`: whether to remove the combined event file after splitting.
- `event_based_logging_output_separator`: separator used for writing the events.
- `event_based_logging_output_parser`: parsing function to handle the output of binary_c. There is a function already present for this, so no need to provide this yourself unless you have special requests.
- `event_based_logging_parameter_list_dict`: dictionary that contains the parameter name list for each specific event (see [the events based logging page in the documentation](https://binary_c.gitlab.io/binary_c-python/event_based_logging_descriptions.html)). The current present dictionary is designed to handle the events that are present in this release, but if you add your own events you need to update this dictionary or provide a custom one.

%% Cell type:markdown id:4a0bc9b2-c976-4688-9b2d-9e2d49b17776 tags:

## Example usage
We now show some example usage of the event-based logging. We start as usual with some imports and setting up a population object

%% Cell type:code id:cda61339-be72-4b61-b307-598b66fdfa47 tags:

``` python
import os

from binarycpython.utils.custom_logging_functions import temp_dir
from binarycpython import Population

TMP_DIR = temp_dir("notebooks", "notebook_events_based_logging")
data_dir = os.path.join(TMP_DIR, 'data_dir')
os.makedirs(data_dir, exist_ok=True)

event_based_logging_population = Population(tmp_dir=TMP_DIR)
event_based_logging_population = Population(tmp_dir=TMP_DIR, verbosity=2)
```

%% Cell type:markdown id:75cd9bcf-d480-441c-aba1-21ac6c5d697c tags:

Lets configure the population object to use events based logging

%% Cell type:code id:97382213-2751-4841-a8cd-f0e9e543af7e tags:

``` python
# Set population object
event_based_logging_population.set(
    num_cores=2,
    data_dir=data_dir,
    # binary-c options related to event-based logging
    event_based_logging_enable_event_logging=1,
    event_based_logging_enable_SN_event_logging=1,
    event_based_logging_enable_RLOF_event_logging=1,
    # binary_c-python options related to event-based logging
    event_based_logging_handle_output=True,
    event_based_logging_combine_individual_event_files=True,
    event_based_logging_remove_individual_event_files_after_combining=False,
    event_based_logging_split_events_file_to_each_type=False,
    event_based_logging_remove_original_combined_events_file_after_splitting=False,
)
```

%% Output

    adding: num_cores=2 to population_options
    <<<< Warning: Key does not match previously known parameter:                     adding: data_dir=/tmp/binary_c_python-david/notebooks/notebook_events_based_logging/data_dir to custom_options >>>>
    adding: event_based_logging_enable_event_logging=1 to BSE_options
    adding: event_based_logging_enable_SN_event_logging=1 to BSE_options
    adding: event_based_logging_enable_RLOF_event_logging=1 to BSE_options
    adding: event_based_logging_handle_output=True to population_options
    adding: event_based_logging_combine_individual_event_files=True to population_options
    adding: event_based_logging_remove_individual_event_files_after_combining=False to population_options
    adding: event_based_logging_split_events_file_to_each_type=False to population_options
    adding: event_based_logging_remove_original_combined_events_file_after_splitting=False to population_options

%% Cell type:markdown id:25f02c5f-850e-4259-aa13-a1613c323b58 tags:

And lets provide some systems that can generate us some events. We use a set list of systems through the source-file sampling functionality but that is only for this example. For a more serious sampling you can use e.g. the grid-based sampling (see [grid-based sampling notebook](https://binary_c.gitlab.io/binary_c-python/examples/notebook_population.html))

%% Cell type:code id:34d49455-9b34-4be0-8d64-c6e70cd3e1d4 tags:

``` python
# Configure the source-file sampling
system_dict_test_list = [
    {"M_1": 10},
    {"M_1": 10.0, "M_2": 0.1, "orbital_period": 1000000000},
    {"M_1": 1, "M_2": 0.5, "orbital_period": 100.0},
]

# Create file that contains the systems
source_file_sampling_filename = os.path.join(
    TMP_DIR, "source_file_sampling_filename.txt"
)

# write the source file
with open(source_file_sampling_filename, "w") as f:
    # Loop over system dict
    for system_dict_test_entry in system_dict_test_list:
        argline = " ".join(
            [
                "{} {}".format(key, val)
                for key, val in system_dict_test_entry.items()
            ]
        )
        f.write(argline + "\n")

# Update setting
event_based_logging_population.set(
    source_file_sampling_type="command",
    source_file_sampling_filename=source_file_sampling_filename,
    evolution_type="source_file"
)
```

%% Output

    <<<< Warning: Key does not match previously known parameter:                     adding: source_file_sampling_type=command to custom_options >>>>
    <<<< Warning: Key does not match previously known parameter:                     adding: source_file_sampling_filename=/tmp/binary_c_python-david/notebooks/notebook_events_based_logging/source_file_sampling_filename.txt to custom_options >>>>
    adding: evolution_type=source_file to population_options

%% Cell type:markdown id:8d3b08d8-6f5e-4482-a494-147cd5db3f93 tags:

Lets now run these systems and explore the contents of the event files.

%% Cell type:code id:d0939590-c5b2-4c3e-b246-efee4ea87c16 tags:

``` python
event_based_logging_population.evolve()
```

%% Output

    Do join of subprocesses ...
    Joined all subprocesses.
    **********************************************************
    *  Population-434943c577994ee6ab6354720ea32ede finished! *
    *               The total probability is 3.              *
    *  It took a total of 0.52s to run 3 systems on 2 cores  *
    *                   = 1.05s of CPU time.                 *
    *              Maximum memory use 327.305 MB             *
    **********************************************************
    
    No failed systems were found in this run.
    Do analytics
    Added analytics to metadata
    source file cleanup

    {'population_id': '434943c577994ee6ab6354720ea32ede',
     'evolution_type': 'source_file',
     'failed_count': 0,
     'failed_prob': 0,
     'failed_systems_error_codes': [],
     'errors_exceeded': False,
     'errors_found': False,
     'total_probability': 3,
     'total_count': 3,
     'start_timestamp': 1675290130.381175,
     'end_timestamp': 1675290130.904902,
     'time_elapsed': 0.5237269401550293,
     'total_mass_run': 21.6,
     'total_probability_weighted_mass_run': 21.6,
     'zero_prob_stars_skipped': 0}
    Warning: No parse function set. Make sure you intended to do this.

    ---------------------------------------------------------------------------
    KeyError                                  Traceback (most recent call last)
Input     In [8], in <cell line: 1>()
    ----> 1 event_based_logging_population.evolve()
File     ~/.pyenv/versions/3.9.9/envs/binarycpython3.9.9/lib/python3.9/site-packages/binarycpython/utils/population_class.py:488, in Population.evolve(self)
        485     return
        487 # Execute population evolution subroutines
    --> 488 result = self._evolve_population_wrapper()
        489 if result is False:
        490     print("Error detected in _evolve_population() : stopping here")
File     ~/.pyenv/versions/3.9.9/envs/binarycpython3.9.9/lib/python3.9/site-packages/binarycpython/utils/population_extensions/evolution_functions.py:61, in evolution_functions._evolve_population_wrapper(self)
         51 """
         52 Function to evolve populations. This handles the setting up, evolving
         53 and cleaning up of a population of stars.
         54
         55 Returns True if the grid runs, False on error.
         56 """
         58 ############################################################
         59 # Prepare code/initialise grid.
         60 # set custom logging, set up store_memaddr, build grid code. dry run grid code.
    ---> 61 if self._setup() is False:
         62     return
         64 ############################################################
         65 # Evolve systems
File     ~/.pyenv/versions/3.9.9/envs/binarycpython3.9.9/lib/python3.9/site-packages/binarycpython/utils/population_class.py:580, in Population._setup(self)
        578 # Source file
        579 elif self.population_options["evolution_type"] == "source_file":
    --> 580     self._source_file_sampling_setup()
        582 # Monte-carlo
        583 elif self.population_options["evolution_type"] == "monte_carlo":
File     ~/.pyenv/versions/3.9.9/envs/binarycpython3.9.9/lib/python3.9/site-packages/binarycpython/utils/population_extensions/source_file_sampling.py:173, in source_file_sampling._source_file_sampling_setup(self)
        168 """
        169 setup function for the source file sampling evolution method
        170 """
        172 # check if file exists
    --> 173 if not os.path.isfile(self.population_options["source_file_sampling_filename"]):
        174     self.vb_critical("Source file doesnt exist")
        176 # check choice of sampling_file type:
    KeyError: 'source_file_sampling_filename'

%% Cell type:code id:37a47d9b-3b71-4d65-89f9-f76797558b0f tags:

``` python
combined_events_filename = os.path.join(data_dir, event_based_logging_population.population_options['event_based_logging_combined_events_filename'])
with open(combined_events_filename, 'r') as f:
    combined_events = f.readlines()
```

%% Cell type:code id:919cae2b-1738-4dd6-bdad-0530e3bf0fea tags:

``` python
for event in combined_events:
    print(event)
```

%% Output

    C77ECB9F-43C3-4CE8-BFE1-CA98D8FE2F73	SN_BINARY	10	0.1	1e+09	9.09806e+06	0	0.02	1	48267	1.33487	13	28.4842	14	0	0	-1	3.23995e+06	-18.1859	9.18507	5	727.198	2.81957	1.82729	2.81957	0.00534839	0	3.23834e+06	9.89406e+06	0.1	0.134597	0	0	1	122.788	1.76653	0.193624
    
    1B45A1CD-5E41-49AC-AD7F-7DC1F961D21B	SN_SINGLE	10	0.02	1	1423	1.33469	13	28.4838	14	0	0	9.1865	5	724.338	2.81957	1.82533	2.81957	0.00539487	0	1	356.317	0.0669739	0.324305
    
    16A1CB91-D6D8-4F92-860D-28C0DAA1A8CB	RLOF	1	0.5	100	103.802	0	0.02	1	77405	0.970542	0.50131	41.8386	0.468712	95.7159	0.244766	3	0	77741.6	3	1	0	1.231494317672e+10	0	0.337181	0.50131	0.016932	0.468712	2.79234	0.00161588	10	0	6111.9	-1	0	1	1.231494440429e+10	0	0	0	0	0	0	1
    

%% Cell type:markdown id:5e0b148f-4460-441b-9039-be9c9b25c8d2 tags:

As we see above, we now have some events that we can analyse. The parameters contained in each of these lines are described in [the events based logging page in the documentation](https://binary_c.gitlab.io/binary_c-python/event_based_logging_descriptions.html).
As we see above, we now have some events that we can analyse.

We can parse the contents of each of the events with the `event_based_logging_parameter_list_dict`.

%% Cell type:code id:9504ed65-4fa1-4062-ac42-720ad8207d12 tags:

``` python
def parse_events(events_list, parsing_dict):
    """
    Function to parse the output of the evolution of the system and create a dictionary containing the
    """

    parsed_events_list = []

    # Loop over output
    for event in events_list:
        if line.startswith("EVENT"):
            # Parse output and create dictionary
            parameter_values = line.split()[1:]
            event_type = parameter_values[1]
            parameter_names = parsing_dict[event_type]
            event_dict = {parameter_name: parameter_value for (parameter_name, parameter_value) in zip(parameter_names, parameter_values)}

            # recast values
            event_dict = recast_values(event_dict=event_dict)

            #
            events_list.append(event_dict)

    return events_list

Next up is to run a larger population, so we can make use of the uuid's to relate certain events.
event_based_logging_parameter_list_dict = event_based_logging_population.population_options['event_based_logging_parameter_list_dict']

parsed_events = parse_events(events_list=combined_events, parsing_dict=event_based_logging_parameter_list_dict)
```

%% Output

    ---------------------------------------------------------------------------
    NameError                                 Traceback (most recent call last)
Input     In [2], in <cell line: 25>()
         21             events_list.append(event_dict)
         23     return events_list
    ---> 25 event_based_logging_parameter_list_dict = BHBH_pop.population_options['event_based_logging_parameter_list_dict']
         27 parsed_events = parse_events(events_list=combined_events, parsing_dict=event_based_logging_parameter_list_dict)
    NameError: name 'BHBH_pop' is not defined

%% Cell type:markdown id:53add532-71de-4fcb-907a-62b13ff3cef7 tags:

The parameters contained in each of these lines are described in [the events based logging page in the documentation](https://binary_c.gitlab.io/binary_c-python/event_based_logging_descriptions.html).

%% Cell type:markdown id:c533e011-89ef-42b6-860b-d7c282aa7bfa tags:

The next step would be to run a larger population and log all the events of interest. We can then use the UUID's to cross match certain different events with each other and perform (complex) queries to select e.g. the BHBH DCO_FORMATION events that have experienced at least one pulsational pair-instability supernova.

%% Cell type:code id:1a8f9e60-95da-4768-af24-e2fd14b0f0c6 tags:

``` python
```