21.40 mongodb分片测试

可莉
• 阅读 576

21.40 mongodb分片测试

分片搭建–测试

  • 登录任何一台20000端口
    mongo --port 20000
    use admin
    db.runCommand({ enablesharding : "testdb"}) 或者
    sh.enableSharding("testdb") //指定要分片的数据库
    db.runCommand( { shardcollection : "testdb.table1",key : {id: 1} } ) 或者
    sh.shardCollection("testdb.table1",{"id":1} ) //#指定数据库里需要分片的集合和片键
    use testdb
    for (var i = 1; i <= 10000; i++) db.table1.save({id:i,"test1":"testval1"})//插入测试数据
    db.table1.stats()//查看table1状态

    [root@Dasoncheng ~]# mongo --port 20000 MongoDB shell version v3.4.9 connecting to: mongodb://127.0.0.1:20000/ MongoDB server version: 3.4.9 mongos> use admin switched to db admin mongos> sh.enableSharding("testdb")
    ##指定要分片的数据库testdb,没有回自动创建; { "ok" : 1 } mongos> sh.shardCollection("testdb.table1",{"id":1} )
    ##指定数据库里需要分片的集合和片键 { "collectionsharded" : "testdb.table1", "ok" : 1 } mongos> use testdb switched to db testdb ##下面插入一条数据; mongos> for (var i = 1; i <= 10000; i++) db.table1.save({id:i,"test1":"testval1"})

    WriteResult({ "nInserted" : 1 }) mongos> show dbs admin 0.000GB config 0.001GB testdb 0.000GB mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("59e957b4b1833892fcc37ec7") } shards: { "_id" : "shard1", "host" : "shard1/192.168.60.11:27001,192.168.60.12:27001", "state" : 1 } { "_id" : "shard2", "host" : "shard2/192.168.60.12:27002,192.168.60.13:27002", "state" : 1 } { "_id" : "shard3", "host" : "shard3/192.168.60.11:27003,192.168.60.13:27003", "state" : 1 } active mongoses: "3.4.9" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Balancer lock taken at Fri Oct 20 2017 14:44:12 GMT+0800 (CST) by ConfigServer:Balancer Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "testdb", "primary" : "shard1", "partitioned" : true } testdb.table1 shard key: { "id" : 1 } unique: false balancing: true chunks: shard1 1 ##看这里,存在shard1里面 { "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) mongos> sh.enableSharding("db2") ##再创建库和表,查看存在哪个shard里面; { "ok" : 1 } mongos> sh.shardCollection("db2.cl2",{"id":1} ) { "collectionsharded" : "db2.cl2", "ok" : 1 } mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("59e957b4b1833892fcc37ec7") } shards: { "_id" : "shard1", "host" : "shard1/192.168.60.11:27001,192.168.60.12:27001", "state" : 1 } { "_id" : "shard2", "host" : "shard2/192.168.60.12:27002,192.168.60.13:27002", "state" : 1 } { "_id" : "shard3", "host" : "shard3/192.168.60.11:27003,192.168.60.13:27003", "state" : 1 } active mongoses: "3.4.9" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Balancer lock taken at Fri Oct 20 2017 14:44:12 GMT+0800 (CST) by ConfigServer:Balancer Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "testdb", "primary" : "shard1", "partitioned" : true } testdb.table1 shard key: { "id" : 1 } unique: false balancing: true chunks: shard1 1 { "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) { "_id" : "db2", "primary" : "shard2", "partitioned" : true } db2.cl2 shard key: { "id" : 1 } unique: false balancing: true chunks: shard2 1 ##看这里,存在shard2里面 { "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard2 Timestamp(1, 0) ##分片分布在各个shard里面,测试成功;

21.41 mongodb备份恢复

  • 备份所有库
    mongodump --host 127.0.0.1 --port 20000 -o /tmp/mongobak/alldatabase

  • 备份指定库
    mongodump --host 127.0.0.1 --port 20000 -d mydb -o /tmp/mongobak
    它会在/tmp/目录下面生成一个mydb的目录

  • 指定备份集合
    mongodump --host 127.0.0.1 --port 20000 -d mydb -c c1 -o /tmp/mongobak/
    它依然会生成mydb目录,再在这目录下面生成两个文件

  • 导出集合为json文件
    mongoexport --host 127.0.0.1 --port 20000 -d mydb -c c1 -o /tmp/mydb2/1.json

    ##备份所有库: [root@Dasoncheng ~]# mongodump --host 127.0.0.1 --port 20000 -o /tmp/mongobak/alldatabase
    ##备份所有库;-o指定文件夹,如果没有回自动创建 2017-10-20T17:00:15.154+0800 writing admin.system.version to 2017-10-20T17:00:15.160+0800 done dumping admin.system.version (1 document) 2017-10-20T17:00:15.160+0800 writing testdb.table1 to 2017-10-20T17:00:15.160+0800 writing config.lockpings to 2017-10-20T17:00:15.161+0800 writing config.changelog to 2017-10-20T17:00:15.161+0800 writing config.locks to 2017-10-20T17:00:15.169+0800 done dumping config.lockpings (13 documents) 2017-10-20T17:00:15.169+0800 writing config.shards to 2017-10-20T17:00:15.178+0800 done dumping config.shards (3 documents) 2017-10-20T17:00:15.178+0800 writing config.collections to 2017-10-20T17:00:15.192+0800 done dumping config.collections (2 documents) 2017-10-20T17:00:15.192+0800 writing config.databases to 2017-10-20T17:00:15.195+0800 done dumping config.changelog (7 documents) 2017-10-20T17:00:15.195+0800 writing config.chunks to 2017-10-20T17:00:15.203+0800 done dumping config.locks (5 documents) 2017-10-20T17:00:15.203+0800 writing config.mongos to 2017-10-20T17:00:15.203+0800 done dumping config.chunks (2 documents) 2017-10-20T17:00:15.203+0800 writing config.version to 2017-10-20T17:00:15.206+0800 done dumping config.version (1 document) 2017-10-20T17:00:15.206+0800 writing config.tags to 2017-10-20T17:00:15.209+0800 done dumping config.tags (0 documents) 2017-10-20T17:00:15.209+0800 writing config.migrations to 2017-10-20T17:00:15.213+0800 done dumping config.migrations (0 documents) 2017-10-20T17:00:15.213+0800 writing db2.cl2 to 2017-10-20T17:00:15.215+0800 done dumping config.databases (2 documents) 2017-10-20T17:00:15.226+0800 done dumping config.mongos (2 documents) 2017-10-20T17:00:15.227+0800 done dumping db2.cl2 (0 documents) 2017-10-20T17:00:15.260+0800 done dumping testdb.table1 (10000 documents) [root@Dasoncheng ~]# ll /tmp/mongobak/alldatabase/ total 4 drwxr-xr-x 2 root root 69 Oct 20 17:00 admin drwxr-xr-x 2 root root 4096 Oct 20 17:00 config drwxr-xr-x 2 root root 47 Oct 20 17:00 db2 drwxr-xr-x 2 root root 53 Oct 20 17:00 testdb [root@Dasoncheng ~]# ll /tmp/mongobak/alldatabase/testdb/ total 532 -rw-r--r-- 1 root root 540000 Oct 20 17:00 table1.bson
    ##这个文件是主要数据.bson -rw-r--r-- 1 root root 145 Oct 20 17:00 table1.metadata.json ##这个文件是建表信息.json

    ##备份指定库 [root@Dasoncheng ~]# mongodump --host 127.0.0.1 --port 20000 -d testdb -o /tmp/mongobak
    ##-d指定库;-o指定备份文件夹 没有则自动创建; 2017-10-20T16:56:15.484+0800 writing testdb.table1 to 2017-10-20T16:56:15.633+0800 done dumping testdb.table1 (10000 documents) [root@Dasoncheng ~]# ll /tmp/mongobak/testdb/ ##因为这个库只有一个集合testdb,所以 total 532 -rw-r--r-- 1 root root 540000 Oct 20 16:56 table1.bson -rw-r--r-- 1 root root 145 Oct 20 16:56 table1.metadata.json

    ##指定备份集合 [root@Dasoncheng ~]# mongodump --host 127.0.0.1 --port 20000 -d testdb -c table1 -o /tmp/mongobak2 2017-10-20T16:59:03.087+0800 writing testdb.table1 to 2017-10-20T16:59:03.166+0800 done dumping testdb.table1 (10000 documents) [root@Dasoncheng ~]# ll /tmp/mongobak2/testdb/ total 532 -rw-r--r-- 1 root root 540000 Oct 20 16:59 table1.bson -rw-r--r-- 1 root root 145 Oct 20 16:59 table1.metadata.json

    ##导出集合为json文件 [root@Dasoncheng ~]# mongoexport --host 127.0.0.1 --port 20000 -d testdb -c table1 -o /tmp/testdb/1.json 2017-10-20T17:02:09.210+0800 connected to: 127.0.0.1:20000 2017-10-20T17:02:09.593+0800 exported 10000 records [root@Dasoncheng ~]# ll /tmp/testdb/1.json -rw-r--r-- 1 root root 748894 Oct 20 17:02 /tmp/testdb/1.json [root@Dasoncheng ~]# tail /tmp/testdb/1.json {"_id":{"$oid":"59e9b0bb4bdfc034c89123b2"},"id":9991.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123b3"},"id":9992.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123b4"},"id":9993.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123b5"},"id":9994.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123b6"},"id":9995.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123b7"},"id":9996.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123b8"},"id":9997.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123b9"},"id":9998.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123ba"},"id":9999.0,"test1":"testval1"} {"_id":{"$oid":"59e9b0bb4bdfc034c89123bb"},"id":10000.0,"test1":"testval1"}

MongoDB恢复:

  • 恢复所有库
    mongorestore -h 127.0.0.1 --port 20000 --drop dir/ //其中dir是备份所有库的目录名字,其中--drop可选,意思是当恢复之前先把之前的数据删除,不建议使用

  • 恢复指定库
    mongorestore -d mydb dir/ //-d跟要恢复的库名字,dir就是该库备份时所在的目录

  • 恢复集合
    mongorestore -d mydb -c testc dir/mydb/testc.bson // -c后面跟要恢复的集合名字,dir是备份mydb库时生成文件所在路径,这里是一个bson文件的路径

  • 导入集合
    mongoimport -d mydb -c testc --file /tmp/testc.json

    ##回复所有库; [root@Dasoncheng ~]# ls /tmp/mongobak/alldatabase/ ##备份时指定的文件夹; admin config db2 testdb [root@Dasoncheng ~]# mongo --port 20000 MongoDB shell version v3.4.9 connecting to: mongodb://127.0.0.1:20000/ MongoDB server version: 3.4.9 mongos> show dbs admin 0.000GB config 0.001GB db2 0.000GB testdb 0.000GB mongos> use db2 switched to db db2 mongos> db.dropDatabase() { "dropped" : "db2", "ok" : 1 } mongos> use testdb switched to db testdb mongos> db.dropDatabase() { "dropped" : "testdb", "ok" : 1 } mongos> show dbs admin 0.000GB config 0.001GB [root@Dasoncheng ~]# mongorestore -h 127.0.0.1 --port 20000 /tmp/mongobak/alldatabase
    ##还原的时候使用备份的文件夹即可; 2017-10-20T17:50:44.598+0800 preparing collections to restore from 2017-10-20T17:50:44.599+0800 Failed: cannot do a full restore on a sharded system - remove the 'config' directory from the dump directory first
    ##这里报错,config库不能还原(config库很重要 不能随便还原),提示删除 我们接下来删除; [root@Dasoncheng ~]# rm -rf /tmp/mongobak/alldatabase/config/ [root@Dasoncheng ~]# mongorestore -h 127.0.0.1 --port 20000 /tmp/mongobak/alldatabase 2017-10-20T17:51:20.512+0800 preparing collections to restore from 2017-10-20T17:51:20.517+0800 reading metadata for testdb.table1 from /tmp/mongobak/alldatabase/testdb/table1.metadata.json 2017-10-20T17:51:20.522+0800 reading metadata for db2.cl2 from /tmp/mongobak/alldatabase/db2/cl2.metadata.json 2017-10-20T17:51:20.587+0800 restoring testdb.table1 from /tmp/mongobak/alldatabase/testdb/table1.bson 2017-10-20T17:51:20.604+0800 restoring db2.cl2 from /tmp/mongobak/alldatabase/db2/cl2.bson 2017-10-20T17:51:20.616+0800 restoring indexes for collection db2.cl2 from metadata 2017-10-20T17:51:20.644+0800 finished restoring db2.cl2 (0 documents) 2017-10-20T17:51:21.876+0800 restoring indexes for collection testdb.table1 from metadata 2017-10-20T17:51:22.105+0800 finished restoring testdb.table1 (10000 documents) 2017-10-20T17:51:22.105+0800 done ##还原成功,查看还原后的库: [root@Dasoncheng ~]# mongo --port 20000 mongos> show dbs admin 0.000GB config 0.001GB db2 0.000GB testdb 0.000GB ##老铁,没毛病;接下来恢复库 以及集合

    ##恢复指定库; [root@Dasoncheng ~]# ls /tmp/mongobak/testdb/ table1.bson table1.metadata.json [root@Dasoncheng ~]# mongorestore -d testdb /tmp/mongobak/testdb/ ##指定包含库文件的目录就行; 2017-10-20T17:57:21.077+0800 the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead 2017-10-20T17:57:21.078+0800 building a list of collections to restore from /tmp/mongobak/testdb dir 2017-10-20T17:57:21.081+0800 reading metadata for testdb.table1 from /tmp/mongobak/testdb/table1.metadata.json 2017-10-20T17:57:21.104+0800 restoring testdb.table1 from /tmp/mongobak/testdb/table1.bson 2017-10-20T17:57:22.389+0800 restoring indexes for collection testdb.table1 from metadata 2017-10-20T17:57:22.474+0800 finished restoring testdb.table1 (10000 documents) 2017-10-20T17:57:22.474+0800 done ##还原库没毛病!

    ##恢复集合; [root@Dasoncheng ~]# mongorestore -d testdb -c table1 /tmp/mongobak/testdb/table1.bson …… 2017-10-20T18:00:05.663+0800 restoring indexes for collection testdb.table1 from metadata 2017-10-20T18:00:05.673+0800 finished restoring testdb.table1 (10000 documents) 2017-10-20T18:00:05.673+0800 done

点赞
收藏
评论区
推荐文章
blmius blmius
3年前
MySQL:[Err] 1292 - Incorrect datetime value: ‘0000-00-00 00:00:00‘ for column ‘CREATE_TIME‘ at row 1
文章目录问题用navicat导入数据时,报错:原因这是因为当前的MySQL不支持datetime为0的情况。解决修改sql\mode:sql\mode:SQLMode定义了MySQL应支持的SQL语法、数据校验等,这样可以更容易地在不同的环境中使用MySQL。全局s
皕杰报表之UUID
​在我们用皕杰报表工具设计填报报表时,如何在新增行里自动增加id呢?能新增整数排序id吗?目前可以在新增行里自动增加id,但只能用uuid函数增加UUID编码,不能新增整数排序id。uuid函数说明:获取一个UUID,可以在填报表中用来创建数据ID语法:uuid()或uuid(sep)参数说明:sep布尔值,生成的uuid中是否包含分隔符'',缺省为
待兔 待兔
4个月前
手写Java HashMap源码
HashMap的使用教程HashMap的使用教程HashMap的使用教程HashMap的使用教程HashMap的使用教程22
Jacquelyn38 Jacquelyn38
3年前
2020年前端实用代码段,为你的工作保驾护航
有空的时候,自己总结了几个代码段,在开发中也经常使用,谢谢。1、使用解构获取json数据let jsonData  id: 1,status: "OK",data: 'a', 'b';let  id, status, data: number   jsonData;console.log(id, status, number )
Wesley13 Wesley13
3年前
mysql设置时区
mysql设置时区mysql\_query("SETtime\_zone'8:00'")ordie('时区设置失败,请联系管理员!');中国在东8区所以加8方法二:selectcount(user\_id)asdevice,CONVERT\_TZ(FROM\_UNIXTIME(reg\_time),'08:00','0
Wesley13 Wesley13
3年前
MongoDB 集群设置集合分片生效及查看集合分片情况
一、设计DB分片与Collection分片连接mongos/opt/mongodb/mongodblinuxx86_642.4.8/bin/mongo 127.0.0.1:27017使用admin数据库useadmin指定testdb分片生效db.runCommand({ena
Wesley13 Wesley13
3年前
00:Java简单了解
浅谈Java之概述Java是SUN(StanfordUniversityNetwork),斯坦福大学网络公司)1995年推出的一门高级编程语言。Java是一种面向Internet的编程语言。随着Java技术在web方面的不断成熟,已经成为Web应用程序的首选开发语言。Java是简单易学,完全面向对象,安全可靠,与平台无关的编程语言。
Stella981 Stella981
3年前
Django中Admin中的一些参数配置
设置在列表中显示的字段,id为django模型默认的主键list_display('id','name','sex','profession','email','qq','phone','status','create_time')设置在列表可编辑字段list_editable
Wesley13 Wesley13
3年前
MySQL部分从库上面因为大量的临时表tmp_table造成慢查询
背景描述Time:20190124T00:08:14.70572408:00User@Host:@Id:Schema:sentrymetaLast_errno:0Killed:0Query_time:0.315758Lock_
Python进阶者 Python进阶者
9个月前
Excel中这日期老是出来00:00:00,怎么用Pandas把这个去除
大家好,我是皮皮。一、前言前几天在Python白银交流群【上海新年人】问了一个Pandas数据筛选的问题。问题如下:这日期老是出来00:00:00,怎么把这个去除。二、实现过程后来【论草莓如何成为冻干莓】给了一个思路和代码如下:pd.toexcel之前把这