Skip to content

Changes

Page history
Update SDG classifier authored by Ioanna Mandilara's avatar Ioanna Mandilara
Hide whitespace changes
Inline Side-by-side
SDG-classifier.md
View page @ 49ae5736
| _class_ SDGDetector.SDG_classifier(pretrained_model_path,pretrained_model_name,top_keywords = 5, diversity = 0.3, n_gram_range = (1,2) , sentence_model_name )|
|----------|
|Parameters | Description | Value |
| ------------ | ------------------------------- | --------------- |
|`pretrained_model_name` | The name of the pre-trained model| {'RoBERTa','XLNet'}|
|`pretrained_model_path` | The path of the fine-tuned model| str |
|`top_keywords` | The number of the top keywords of texts| int|
|`diversity` | The diversity ([0,1]) of the top produced keywords. A high diversity score will create very diverse keyphrases/keywords compared with low diversity, which will produce very similar keywords with the respective ones using the cosine similarity method: | float|
|`n_gram_range` | The different lengths of keyphrases| tuple of shape (range_x,range_y)|
|`sentence_model_name ` | The name of the pre-trained sentence model| {'all-mpnet-base-v2','distilbert-base-nli-mean-tokens','all-MiniLM-L6-v2'}|
### Methods
##### predict(text,return_association=False)
Find the most relevant SDGs in the provided texts and computes the association of each text with the SDGs using the pre-trained fine-tuned model and the method of keywords extraction. The formula of the association is
```math
Association = \left\{
\begin{array}{ll}
0.7*probability + 0.3*avg\_cosine\_similarity & \text{,for SDGs 1-16} \\
0.5*avg\_cosine\_similarity & \text{,for SDG 17} \\
\end{array}
\right.
```
The above formula has two branches, because the OSDG community dataset, which have used for the Classification Task includes only texts related to the first 16 SDGs. So, for SDG 17 we get only the results from the Keywords Extraction technique multiplied with a coefficient equals to 0.5.
|Parameters | Description | Returns | Description |
| ------------ | ------------------------------- | ------------ | ------------------------------- |
|`text`: list of length (# of texts) | Input texts |`sdg` : list of length (# of texts) | Returns the the most relevant SDG per text |
| `return_association`: Boolean| If set to True then returns the association of each text with the SDGs |`sdg_name`: list of length (# of texts) | Returns the name of the most relevant SDG per text |
| | |`association`: list of lists (association of length 17) of length (# of texts)| Returns the association of each text with each one of the SDGs.|
### Examples
```python
text = ['Europe has always been the home of industry. For centuries, it has been a pioneer in industrial innovation and has helped \
improve the way people around the world produce, consume and do business. Based on a strong internal market, the European industry \
has long powered our economy, providing a stable living for millions and creating the social hubs around which our communities are built.',
'Many people have a rosy view of the wild. Some think nonhuman animals live in some kind of paradise in the wild. However, animals living \
in nature have lives that are far from idyllic, and most of them have to deal with the reality of constant threat of tremendous suffering. \
Wild animal suffering is widely prevalent. Many people do not see or think about this aspect of living in nature. Others believe that wild animal \
suffering is not a serious issue because animals can cope with their suffering better than domesticated animals. Most of us may accept that nonhuman \
animals experience suffering, yet it may be easy to think they suffer less than they really do. This, however is not so.If sentient beings matter, \
we shouldn’t be indifferent towards wild animal suffering. Moreover, we should bear in mind that the amount of suffering and premature death \
present in nature is very significant. Wild animal suffering is much more prevalent than many people think. Most animals die in painful ways \
when they are very young, so they have no chances for enjoyment, for pursuing their aims or fulfilling their capacities. This is so mainly due to \
the reproductive strategies prevalent in nature due to evolutionary reasons. The huge populations of animals living in the wild also make this issue \
especially pressing. The following texts explain this in detail.']
```
```python
from SDGDetector import SDGDetector
combo = SDGDetector.SDG_classifier(pretrained_model_name='xlnet',pretrained_model_path='../SDGDetector/model/xlnet_model',
sentence_model_name='all-mpnet-base-v2',top_keywords=10,diversity=0.3,n_gram_range=(1,2))
sdg,sdg_name,association = combo.predict(text,return_association=True)
for i in range(len(text)):
print('The most relevant SDG of the provided text {}: {}'.format(i+1,sdg_name[i]))
print('======== The the association of the text with the SDGs: ========')
for j in range(0,16):
print('Goal {}: {:.2f}'.format(j+1,association[i][j]))
print()
| _class_ SDGDetector.SDG_classifier(pretrained_model_path,pretrained_model_name,top_keywords = 5, diversity = 0.3, n_gram_range = (1,2) , sentence_model_name )|
|----------|
|Parameters | Description | Value |
| ------------ | ------------------------------- | --------------- |
|`pretrained_model_name` | The name of the pre-trained model| {'RoBERTa','XLNet'}|
|`pretrained_model_path` | The path of the fine-tuned model| str |
|`top_keywords` | The number of the top keywords of texts| int|
|`diversity` | The diversity ([0,1]) of the top produced keywords. A high diversity score will create very diverse keyphrases/keywords compared with low diversity, which will produce very similar keywords with the respective ones using the cosine similarity method: | float|
|`n_gram_range` | The different lengths of keyphrases| tuple of shape (range_x,range_y)|
|`sentence_model_name ` | The name of the pre-trained sentence model| {'all-mpnet-base-v2','distilbert-base-nli-mean-tokens','all-MiniLM-L6-v2'}|
### Methods
##### predict(text,return_association=False)
Find the most relevant SDGs in the provided texts and computes the association of each text with the SDGs using the pre-trained fine-tuned model and the method of keywords extraction. The formula of the association is
```math
{r}_{SDG} = \left\{
\begin{array}{ll}
0.7*probability + 0.3*avg\_cosine\_similarity & \text{,for SDGs 1-16} \\
0.5*avg\_cosine\_similarity & \text{,for SDG 17} \\
\end{array}
\right.
```
The above formula has two branches, because the OSDG community dataset, which have used for the Classification Task includes only texts related to the first 16 SDGs. So, for SDG 17 we get only the results from the Keywords Extraction technique multiplied with a coefficient equals to 0.5.
|Parameters | Description | Returns | Description |
| ------------ | ------------------------------- | ------------ | ------------------------------- |
|`text`: list of length (# of texts) | Input texts |`sdg` : list of length (# of texts) | Returns the the most relevant SDG per text |
| `return_association`: Boolean| If set to True then returns the association of each text with the SDGs |`sdg_name`: list of length (# of texts) | Returns the name of the most relevant SDG per text |
| | |`association`: list of lists (association of length 17) of length (# of texts)| Returns the association of each text with each one of the SDGs.|
### Examples
```python
text = ['Europe has always been the home of industry. For centuries, it has been a pioneer in industrial innovation and has helped \
improve the way people around the world produce, consume and do business. Based on a strong internal market, the European industry \
has long powered our economy, providing a stable living for millions and creating the social hubs around which our communities are built.',
'Many people have a rosy view of the wild. Some think nonhuman animals live in some kind of paradise in the wild. However, animals living \
in nature have lives that are far from idyllic, and most of them have to deal with the reality of constant threat of tremendous suffering. \
Wild animal suffering is widely prevalent. Many people do not see or think about this aspect of living in nature. Others believe that wild animal \
suffering is not a serious issue because animals can cope with their suffering better than domesticated animals. Most of us may accept that nonhuman \
animals experience suffering, yet it may be easy to think they suffer less than they really do. This, however is not so.If sentient beings matter, \
we shouldn’t be indifferent towards wild animal suffering. Moreover, we should bear in mind that the amount of suffering and premature death \
present in nature is very significant. Wild animal suffering is much more prevalent than many people think. Most animals die in painful ways \
when they are very young, so they have no chances for enjoyment, for pursuing their aims or fulfilling their capacities. This is so mainly due to \
the reproductive strategies prevalent in nature due to evolutionary reasons. The huge populations of animals living in the wild also make this issue \
especially pressing. The following texts explain this in detail.']
```
```python
from SDGDetector import SDGDetector
combo = SDGDetector.SDG_classifier(pretrained_model_name='xlnet',pretrained_model_path='../SDGDetector/model/xlnet_model',
sentence_model_name='all-mpnet-base-v2',top_keywords=10,diversity=0.3,n_gram_range=(1,2))
sdg,sdg_name,association = combo.predict(text,return_association=True)
for i in range(len(text)):
print('The most relevant SDG of the provided text {}: {}'.format(i+1,sdg_name[i]))
print('======== The the association of the text with the SDGs: ========')
for j in range(0,16):
print('Goal {}: {:.2f}'.format(j+1,association[i][j]))
print()
```
\ No newline at end of file