Simple Search

URL: http://localhost:2354/seearch (Updated)

Issue : #298 (closed)

Search Option :

• Keywords
• Publications
• Collections
• Proveniences
• Periods
• Transliteraton
• IDs

Required Containers :

1. Logstash
2. Elastic Search
3. Kibana (For visualization process, Optional)

How to set up users in ElasticSearch? Refer logstash/README.md

The indices generated by ElasticSearch are dynamic in nature, hence they can be excluded. But to index them it may take a minimum of 20 mins to 1 hour depending on the machine.

Additionally:

1. Now we can set result per page.
2. Layout from Compact to Expanded and vice versa supported.
3. Pagination added.

PS:

1. Filter options are not addressed in this PR.

marked the checklist item Publications as completed

added 1 commit

• d9ffd03e - Simple Search for publication added. Fixed search result view for different layouts.

Compare with previous version

added 1 commit

• 5bbdaacd - Unloading /elastic_search/data/

Compare with previous version

Just as a note, this breaks the Bibliography API. Fixing it would mean updating CakePHP to 3.9 (or higher).

That was some temporary changes to work with cURL.

@amansheaven Any update on this?

yeah, I have a dockerfile which enables cURL, you can promote this change with your merge attaching them to the post

FROM alpine:3.11
ARG version
ARG apk_repo
ARG php_version
LABEL description="Alpine-based php-fpm image for CDLI Cake"
LABEL maintainer="Cuneiform Digital Library Initiative <cdli.systems@gmail.com>"
LABEL version="$version"
ENV PHP_VERSION=$php_version
ADD https://dl.bintray.com/php-alpine/key/php-alpine.rsa.pub /etc/apk/keys/php-alpine.rsa.pub
RUN set -ex; \
  addgroup -g 2354 -S cdli; \
  adduser -u 2354 -D -S -G cdli cdli; \
  echo "https://dl.bintray.com/php-alpine/v3.11/php-7.4" >> /etc/apk/repositories;
RUN apk update;
RUN apk add --update \
    ca-certificates \
    php-fpm \
    php-ctype \
	php-intl \
	php-json \
    php-mbstring \
    php-mysqli \
    php-mysqlnd \
    php-opcache \
    php-pdo \
    php-pdo_mysql \
    php-session \
    php-xml \
    php-curl \
    php-redis \
    php-apcu; \
  \
  rm -rf /var/cache/apk/* /tmp/pear ~/.pearrc;
RUN apk add --update curl libcurl;
# Sane defaults
RUN set -ex; \
  sed -ri \
    -e 's|^;?expose_php = .*|expose_php = off|' \
    -e 's|^;?allow_url_fopen = .*|allow_url_fopen = off|' \
    -e 's|^;?opcache.error_log=.*|opcache.error_log = /proc/self/fd/2|' \
    -e 's|^;?opcache.fast_shutdown=.*|opcache.fast_shutdown = 1|' \
    /etc/php7/php.ini; \
  \
  sed -ri \
    -e 's|^;?error_log = .*|error_log = /proc/self/fd/2|' \
    -e 's|^;?emergency_restart_threshold = .*|emergency_restart_threshold = 4|' \
    -e 's|^;emergency_restart_interval = .*|emergency_restart_interval = 1m|' \
    -e 's|^;?process_control_timeout = .*|process_control_timeout = 10s|' \
    -e 's|^;?daemonize = .*|daemonize = no|' \
    -e 's|^;?events.mechanism = .*|events.mechanism = epoll|' \
    /etc/php7/php-fpm.conf; \
  \
  sed -ri \
    -e 's|^;?user = .*|user = cdli|' \
    -e 's|^;?group = .*|group = cdli|' \
    -e 's|^;?listen = .*|listen = 9000|' \
    -e 's|^;?pm = .*|pm = dynamic|' \
    -e 's|^;?pm.max_children = .*|pm.max_children = 24|' \
    -e 's|^;?pm.start_servers = .*|pm.start_servers = 8|' \
    -e 's|^;?pm.min_spare_servers = .*|pm.min_spare_servers = 4|' \
    -e 's|^;?pm.max_spare_servers = .*|pm.max_spare_servers = 12|' \
    -e 's|^;?pm.max_requests = .*|pm.max_requests = 250|' \
    -e 's|^;?pm.status_path = .*|pm.status_path = /status|' \
    -e 's|^;?access.log = .*|access.log = /proc/self/fd/2|' \
    -e 's|^;?access.format = .*|access.format = "%R - %u %t \"%m %r%Q%q\" %s"|' \
    -e 's|^;?catch_workers_output = .*|catch_workers_output = yes|' \
    /etc/php7/php-fpm.d/www.conf;
# RUN echo "extension=redis.so" >> /etc/php7/php.ini;
WORKDIR /srv
CMD ["/usr/sbin/php-fpm7"]

this would be dockerfile for new app cake to make docker use this file instead of registry this change needs to be induced

app_cake:
    # image: registry.gitlab.com/cdli/framework/app_cake:0.4.0
    build:
      context: ../app/cake
      dockerfile: Dockerfile

these changes move to a higher alpine version and new repo location this also lead to increase in a minor version of PHP as far as I am using it does not have problem but we need to keep an eye on it

I don't think enabling cURL requires disabling allow_url_fopen. Either way, I don't need allow_url_fopen if we have cURL and CakePHP 3.9+.

oh, in one of the packages i use they have something similar so they just chain http chandlers in a order and whichever is present can be used something like this

cURL || fopen || {custom_handler}

if that can work with the task you can opt for it :)

I was thinking of using https://book.cakephp.org/3/en/core-libraries/httpclient.html which only supports cURL as of 3.7.

Any update? Should I proceed with cURL ?

@larsgw If you are testing this PR, try to access http://localhost:9200/_cat/indices and verify if all the following indices are present :

• artifact
• collection
• editors_publications
• authors_publications
• artifacts_publications
• inscription

If you try to generate indices on your local machine it will be great. You will need to make changes in logstash.conf.

added 29 commits

• 5bbdaacd...cf8ed6cc - 28 commits from branch phoenix/develop
• 21ba6cd1 - Merge branch 'phoenix/develop' into phoenix/feature/search

Compare with previous version

added 1 commit

• 3da2420b - Toggle Layout funtionality added.

Compare with previous version

added 1 commit

• 4ee6ac88 - Pagination + Result Per Page Added

Compare with previous version

changed the description

Shouldn't it be installed automatically when deploying? Since you are not using it I will deactivate it for now so I can test.

assigned to @larsgw

• Choices for type of search have not been updated
• Look into caching
• Add a message saying there were no hits when there are no hits and suggest a random artifact @RoLoDeXx
• My search parameters are not shown anywhere on the search results display

added 1 commit

• f8ac795c - Indices S44YpzrUR_qtxW_ugivtwg, ZeFWm2vbTiG2kzLSMXdysQ, UAnV9mDDT0m03wOpIg_fsA added

Compare with previous version

added 1 commit

• b8b4a8f8 - Indices JA-ipXNVQPmBdLLPbCFuJA, Kwiy6QQ5RieAUMMW_29AYg, 8x_VXZ6YSW6nbskOoqNAdA added

Compare with previous version

added 1 commit

• 1bb38576 - Indices hQX13cC2RRq9xjcnOgz8PA, QhDrP5c7S0K6bftWWPbaQw, TpL-ZXQAT0eaEK95u-3ryA added

Compare with previous version

Choices for type of search have not been updated

I haven't setup search at /search. It is at /seearch

added 1 commit

• 36a97383 - _state, node.lock added

Compare with previous version

Sometimes I see this error : Deprecated: Array and string offset access syntax with curly braces is deprecated in /srv/app/cake/vendor/cakephp/cakephp/src/View/Helper/FormHelper.php on line 2024