Difference: ElasticsearchKibanaAuthenticationNotes (5 vs. 6)

Revision 62014-05-26 - IvanKadochnikov

Line: 1 to 1
 
META TOPICPARENT name="NoSQLStorageResearch"
Line: 15 to 15
 
      • TODO: https, need cooperation from clients (e.g. MIG writer)
    • all ports proxy towards localhost:9200
    • r/o enforced with mod_rewrite on http method + location
Changed:
<
<
* TODO: forbid scripted searches (unsafe) script.disable_dynamic: true
>
>
      • what are sometimes considered "writing" GETs can only flush indices and clear caches. In a strictest sense these are write operations, but not really dangerous.
         /_cache/clear,/_flush,/_optimize,/_refresh,/{index}/_cache/clear,/{index}/_flush,/{index}/_optimize,/{index}/_refresh 
      • "reading" POSTs are provided for compatibility with clients that don't support GETs with payload. Kibana does use POST /_search.
      • we don't make a distinction between administrative POSTs (controlling the cluster) and data write POSTs, write access port provides full access.
  • forbid dynamic scripting
     script.disable_dynamic: true 
  * TODO? index-level separation
Changed:
<
<
rest.action.multi.allow_explicit_index: false

Apache

  • We need Apache for Shibboleth anyway
  • Apache will probably allow cert-based authentication
  • Jetty's authorization is location-based anyway, can be replicated in Apache.
  • At first a simple restriction based on HTTP method is enough:
    • "writing" GETs can only flush indices and clear caches. In a strictest sense these are write operations, but not really dangerous.
       /_cache/clear,/_flush,/_optimize,/_refresh,/{index}/_cache/clear,/{index}/_flush,/{index}/_optimize,/{index}/_refresh 
    • "reading" POSTs are provided for compatibility with clients that don't support GETs with payload. We have predictable clients so we don't need these.
      • actually Kibana does use POST /_search. Location filter will be necessary.
    • in the first pass we don't make a distinction between administrative POSTs (controlling the cluster) and data write POSTs

Jetty plugin

https://github.com/sonian/elasticsearch-jetty/https://github.com/sonian/elasticsearch-jetty/

 /usr/share/elasticsearch/bin/plugin \
                      -url https://oss-es-plugins.s3.amazonaws.com/elasticsearch-jetty/elasticsearch-jetty-1.1.0-beta.zip \
                      -install elasticsearch-jetty 

Problem: Kibana stops working.

XMLHttpRequest cannot load http://dashb-es:9200/_all/_search. Request header field Content-Type is not allowed by Access-Control-Allow-Headers. 
Will need a "real" webserver to solve it with CORS: https://groups.google.com/forum/#!topic/elasticsearch-jetty/L8x3dBM3TEg

Either that, or instead of using downloaded Kibana, put it on the same domain, which also means installing a webserver. Correction: can also use the aimon team method and install a kibana fork as an Elasticsearch plugin:

/usr/share/elasticsearch/bin/plugin -url https://github.com/Pigueiras/kibana/archive/master.zip -i kibana

We would want Apache eventually for SSO integration anyway.

>
>
 rest.action.multi.allow_explicit_index: false 
 
Changed:
<
<

Important steps

>
>

Performance

  • index refresh tweak (for faster bulk indexing, but data appears in searches after index refresh only)
     curl -XPUT localhost:9200/_settings -d '{ "index.refresh_interval": "30s"}' 
  • on nodes half of the memory for java heap, half for cache
  • TODO? bigger nodes?
  • TODO? decouple client and master?
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback