Shield Your Kibana Dashboards

Stella981
• 阅读 478

You work with sensitive data in Elasticsearch indices that you do not want everyone to see in their Kibana dashboards. Like a hospital with patient names. You could give each department their own Elasticsearch cluster in order to prevent all departments to see the patient's names, for example.

But wouldn't it be great if there was only one Elasticsearch cluster and every departments could manage their own Kibana dashboards? And still have the security in place to prevent leaking of private data?

With Elasticsearch Shield, you can create a configurable layer of security on top of your Elasticsearch cluster. In this article, we will explore a small example setup with Shield and Kibana.

In this article, we'll use Elasticsearch 1.4.4, Shield 1.0.1 and Kibana 4.0.0. These are at the time of writing the most-recent  versions. Beginner knowledge of Elasticsearch is expected.

Suppose we want to keep our patient’s names private. An index of patients will be available only for one department. An index of cases will be available to all departments.

First, we'll add some test data to the cluster. Create a file hospital.json with the following content:

{ "index" : { "_index" : "cases", "_type" : "case", "_id" : "101" } }
{ "admission" : "2015-01-03", "discharge" : "2015-01-04", "injury" : "broken arm" }
{ "index" : { "_index" : "cases", "_type" : "case", "_id" : "102" } }
{ "admission" : "2015-01-03", "discharge" : "2015-01-06", "injury" : "broken leg" }
{ "index" : { "_index" : "cases", "_type" : "case", "_id" : "103" } }
{ "admission" : "2015-01-06", "discharge" : "2015-01-07", "injury" : "broken nose" }
{ "index" : { "_index" : "cases", "_type" : "case", "_id" : "104" } }
{ "admission" : "2015-01-07", "discharge" : "2015-01-07", "injury" : "bruised arm" }
{ "index" : { "_index" : "cases", "_type" : "case", "_id" : "105" } }
{ "admission" : "2015-01-08", "discharge" : "2015-01-10", "injury" : "broken arm" }
{ "index" : { "_index" : "patients", "_type" : "patient", "_id" : "101" } }
{ "name" : "Adam", "age" : 28 }
{ "index" : { "_index" : "patients", "_type" : "patient", "_id" : "102" } }
{ "name" : "Bob", "age" : 45 }
{ "index" : { "_index" : "patients", "_type" : "patient", "_id" : "103" } }
{ "name" : "Carol", "age" : 34 }
{ "index" : { "_index" : "patients", "_type" : "patient", "_id" : "104" } }
{ "name" : "David", "age" : 14 }
{ "index" : { "_index" : "patients", "_type" : "patient", "_id" : "105" } }
{ "name" : "Eddie", "age" : 72 }

Then bulk index these documents in the cluster:

$ curl -X POST 'http://localhost:9200/_bulk' --data-binary @./hospital.json

Without security, every user can access all documents. Let's install Shield to add security.

Directions on how to install Shield can be found at the Elasticsearch website . We will do this step by step.

Shield is a commercial product, so first we need to install the license manager:

$ elasticsearch/bin/plugin -i elasticsearch/license/latest
-> Installing elasticsearch/license/latest...
Trying http://download.elasticsearch.org/elasticsearch/license/license-latest.zip...
Downloading .....................................DONE
Installed elasticsearch/license/latest into /home/patrick/blog/elasticsearch/plugins/license

Now install Shield itself in the same manner:

$ elasticsearch/bin/plugin -i elasticsearch/shield/latest
-> Installing elasticsearch/shield/latest...
Trying http://download.elasticsearch.org/elasticsearch/shield/shield-latest.zip...
Downloading .....................................DONE
Installed elasticsearch/shield/latest into /home/patrick/blog/elasticsearch/plugins/shield

You will need to restart the nodes of your cluster to activate the plugins.

In the logs of Elasticsearch you'll see some messages from the new plugins:

[2015-02-12 08:18:01,347][INFO ][shield.license ] [node0] enabling license for [shield]
[2015-02-12 08:18:01,347][INFO ][license.plugin.core ] [node0] license for [shield] - valid
[2015-02-12 08:18:01,355][ERROR][shield.license ] [node0]
#
# Shield license will expire on [Saturday, March 14, 2015]. Cluster health, 
# cluster stats and indices stats operations are blocked on Shield license expiration.
# All data operations (read and write) continue to work. If you have a new license,
# please update it. Otherwise, please reach out to your support contact.
#

Notice that you will get a 30-day trial period to experiment with Shield. Now all our data is protected. See what happens when we try to get a document:

$ curl localhost:9200/cases/case/101?pretty=true
{
    "error" : "AuthenticationException[missing authentication token 
              for REST request [/cases/case/1]]",
    "status" : 401
}

We need to add some roles and users to configure the role-based access control of Shield. First we define the roles and their privileges. The definition of these are found in the elasticsearch/config/shield/roles.yml file. Some roles, like admin and user are predefined:

# All cluster rights
# All operations on all indices
admin:
  cluster: all
  indices:
    '*': all
# Read-only operations on indices
user:
  indices:
    '*': read

Let's edit this roles.yml file to describe our needs. We do not want for every user to access all indices, so we'll change user.indices . We'll add the two roles needed for our organization: doctor and nurse. A doctor has more privileges than a nurse. Doctors can access all indices. Nurses can only access the cases index:

# Read-only operations on indices
#user:
#  indices:
#    '*': read

# Doctors can access all indices
doctor:
  indices:
    '*': read

# Nurses can only access the cases index
nurse:
  indices:
    'cases': read

Now that the roles are defined, we can create users that have these roles. Shield provides three realms to store the users: an internal realm, LDAP or Active Directory. For now, we use the internal realm. The realm is configured in elasticsearch/config/elasticsearch.yml . If nothing is explicitly configured, the internal realm is used.

To add users the esusers command line tool can be used. Let's create two users (both with abc123 as password), one for each role:

$ elasticsearch/bin/shield/esusers useradd alice -r nurse
$ elasticsearch/bin/shield/esusers useradd bob -r doctor

Just to check if the security works:

$ curl --user alice:abc123 localhost:9200/_count?pretty=true
{
  "count" : 5,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  }
}

$ curl --user bob:abc123 localhost:9200/_count?pretty=true
{
  "count" : 10,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  }
}

Alice can see the 5 cases in our cases index. Bob can see those 5 cases plus the 5 patients in the patients index.

Now it's time to add Kibana in the mix. Instructions to download and install Kibana 4 can be found on the Elasticsearch website.

When we start the Kibana server we notice that Kibana is not allowed access to the Elasticsearch cluster:

$ kibana/bin/kibana
{
    "@timestamp": "2015-02-26T08:24:48.958Z",
    "level": "fatal",
    "message": "AuthenticationException[missing authentication token for REST request [/.kibana/config/_search]]",
    "node_env": "production",
    "error": {
        "message": "AuthenticationException[missing authentication token for REST request [/.kibana/config/_search]]",
        "name": "Error",
        "stack": "Error: AuthenticationException[missing authentication token for REST request [/.kibana/config/_search]] 
                 at respond (/home/patrick/kibana/src/node_modules/elasticsearch/src/lib/transport.js:235:15)
                 at checkRespForFailure (/home/patrick/kibana/src/node_modules/elasticsearch/src/lib/transport.js:203:7)
                 at HttpConnector. (/home/patrick/kibana/src/node_modules/elasticsearch/src/lib/connectors/http.js:156:7)
                 at IncomingMessage.bound (/home/patrick/kibana/src/node_modules/elasticsearch/node_modules/lodash-node/modern/internals/baseBind.js:56:17)
                 at IncomingMessage.emit (events.js:117:20)
                 at _stream_readable.js:944:16
                 at process._tickCallback (node.js:442:13)"
    }
}

We need to tell Shield that Kibana is allowed to access our cluster. Extra information of how to let Kibana work with Shield can be found in the Elasticsearch guide .

Shield is shipped with a default configuration for Kibana 4. We find the following role definition in elasticsearch/config/shield/roles.yml.

# The required role for kibana 4 users
kibana4:
  cluster: cluster:monitor/nodes/info
  indices:
    '*':
      - indices:admin/mappings/fields/get
      - indices:admin/validate/query
      - indices:data/read/search
      - indices:data/read/msearch
      - indices:admin/get
    '.kibana':
      - indices:admin/exists
      - indices:admin/mapping/put
      - indices:admin/mappings/fields/get
      - indices:admin/refresh
      - indices:admin/validate/query
      - indices:data/read/get
      - indices:data/read/mget
      - indices:data/read/search
      - indices:data/write/delete
      - indices:data/write/index
      - indices:data/write/update

When the Kibana Server starts it needs to access the .kibana index. So we need to create a user in Shield for Kibana to connect with:

$ elasticsearch/bin/shield/esusers useradd kibana -r kibana4

This account must be configured in Kibana. Modify kibana/conf/kibana.yml :

# If your Elasticsearch is protected with basic auth, this is the user credentials
# used by the Kibana server to perform maintence on the kibana_index at statup. Your Kibana
# users will still need to authenticate with Elasticsearch (which is proxied thorugh
# the Kibana server)
kibana_elasticsearch_username: kibana
kibana_elasticsearch_password: abc123

#----------------------------------------------------------------------------
# In newly version of ElasticSearch 2.1.0, you have to use the following way:
elasticsearch.username: kibana
elasticsearch.password: abc123
#----------------------------------------------------------------------------

The Kibana users must have the kibana4 role to be able to work with Kibana. They must be able to store their visualizations and dashboards in the .kibana index:

$ elasticsearch/bin/shield/esusers roles alice -a kibana4
$ elasticsearch/bin/shield/esusers roles bob -a kibana4

Since the default kibana4 role has read access on all indices, alice and bob will be granted all access on all indices. Therefore the role permissions must be modified:

# Doctors can access all indices
doctor:
  indices:
    'cases,patients':
      - indices:admin/mappings/fields/get
      - indices:admin/validate/query
      - indices:data/read/search
      - indices:data/read/msearch
      - indices:admin/get
      
# Nurses can only access the cases index
nurse:
  indices:
    'cases':
      - indices:admin/mappings/fields/get
      - indices:admin/validate/query
      - indices:data/read/search
      - indices:data/read/msearch
      - indices:admin/get
      
# The required role for kibana 4 users
kibana4:
  cluster:
      - cluster:monitor/nodes/info
      - cluster:monitor/health
  indices:
    '.kibana':
      - indices:admin/exists
      - indices:admin/mapping/put
      - indices:admin/mappings/fields/get
      - indices:admin/refresh
      - indices:admin/validate/query
      - indices:data/read/get
      - indices:data/read/mget
      - indices:data/read/search
      - indices:data/write/delete
      - indices:data/write/index
      - indices:data/write/update
      - indices:admin/create

With this configuration any user with the kibana4 role is able to use Kibana but only sees data that he or she has the proper clearance for.

We can now start the Kibana Server and see that it runs as it should:

$ kibana/bin/kibana
{
    "@timestamp": "2015-02-26T08:53:18.961Z",
    "level": "info",
    "message": "Listening on 0.0.0.0:5601",
    "node_env": "production"
}

We can open a browser and head to localhost:5601 to open the Kibana web interface. Log in as Alice:

Shield Your Kibana Dashboards

After logging in, Kibana will ask for the index pattern. We'll keep it simple:

Shield Your Kibana Dashboards

Then in the discover tab you can add fields to your view. Notice that Alice only sees cases:

Shield Your Kibana Dashboards

When we log in as Bob our discover tab shows both cases and patients:

Shield Your Kibana Dashboards

To summarize: we added security to Elasticsearch with Shield and configured some users and roles. There's nothing more to it!

点赞
收藏
评论区
推荐文章
blmius blmius
2年前
MySQL:[Err] 1292 - Incorrect datetime value: ‘0000-00-00 00:00:00‘ for column ‘CREATE_TIME‘ at row 1
文章目录问题用navicat导入数据时,报错:原因这是因为当前的MySQL不支持datetime为0的情况。解决修改sql\mode:sql\mode:SQLMode定义了MySQL应支持的SQL语法、数据校验等,这样可以更容易地在不同的环境中使用MySQL。全局s
Jacquelyn38 Jacquelyn38
2年前
2020年前端实用代码段,为你的工作保驾护航
有空的时候,自己总结了几个代码段,在开发中也经常使用,谢谢。1、使用解构获取json数据let jsonData  id: 1,status: "OK",data: 'a', 'b';let  id, status, data: number   jsonData;console.log(id, status, number )
Wesley13 Wesley13
2年前
java将前端的json数组字符串转换为列表
记录下在前端通过ajax提交了一个json数组的字符串,在后端如何转换为列表。前端数据转化与请求varcontracts{id:'1',name:'yanggb合同1'},{id:'2',name:'yanggb合同2'},{id:'3',name:'yang
Easter79 Easter79
2年前
swap空间的增减方法
(1)增大swap空间去激活swap交换区:swapoff v /dev/vg00/lvswap扩展交换lv:lvextend L 10G /dev/vg00/lvswap重新生成swap交换区:mkswap /dev/vg00/lvswap激活新生成的交换区:swapon v /dev/vg00/lvswap
皕杰报表之UUID
​在我们用皕杰报表工具设计填报报表时,如何在新增行里自动增加id呢?能新增整数排序id吗?目前可以在新增行里自动增加id,但只能用uuid函数增加UUID编码,不能新增整数排序id。uuid函数说明:获取一个UUID,可以在填报表中用来创建数据ID语法:uuid()或uuid(sep)参数说明:sep布尔值,生成的uuid中是否包含分隔符'',缺省为
Wesley13 Wesley13
2年前
Java获得今日零时零分零秒的时间(Date型)
publicDatezeroTime()throwsParseException{    DatetimenewDate();    SimpleDateFormatsimpnewSimpleDateFormat("yyyyMMdd00:00:00");    SimpleDateFormatsimp2newS
Wesley13 Wesley13
2年前
mysql设置时区
mysql设置时区mysql\_query("SETtime\_zone'8:00'")ordie('时区设置失败,请联系管理员!');中国在东8区所以加8方法二:selectcount(user\_id)asdevice,CONVERT\_TZ(FROM\_UNIXTIME(reg\_time),'08:00','0
Wesley13 Wesley13
2年前
00:Java简单了解
浅谈Java之概述Java是SUN(StanfordUniversityNetwork),斯坦福大学网络公司)1995年推出的一门高级编程语言。Java是一种面向Internet的编程语言。随着Java技术在web方面的不断成熟,已经成为Web应用程序的首选开发语言。Java是简单易学,完全面向对象,安全可靠,与平台无关的编程语言。
Stella981 Stella981
2年前
Django中Admin中的一些参数配置
设置在列表中显示的字段,id为django模型默认的主键list_display('id','name','sex','profession','email','qq','phone','status','create_time')设置在列表可编辑字段list_editable
Wesley13 Wesley13
2年前
MySQL部分从库上面因为大量的临时表tmp_table造成慢查询
背景描述Time:20190124T00:08:14.70572408:00User@Host:@Id:Schema:sentrymetaLast_errno:0Killed:0Query_time:0.315758Lock_
Python进阶者 Python进阶者
2个月前
Excel中这日期老是出来00:00:00,怎么用Pandas把这个去除
大家好,我是皮皮。一、前言前几天在Python白银交流群【上海新年人】问了一个Pandas数据筛选的问题。问题如下:这日期老是出来00:00:00,怎么把这个去除。二、实现过程后来【论草莓如何成为冻干莓】给了一个思路和代码如下:pd.toexcel之前把这