Happywzy

centos系统盘扩容

发表于 2021-12-08 分类于 centos

查看当前系统盘信息

[root@master ~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 7.8G     0  7.8G   0% /dev
tmpfs                    7.8G     0  7.8G   0% /dev/shm
tmpfs                    7.8G  7.4M  7.8G   1% /run
tmpfs                    7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/mapper/centos-root   44G   32G   13G  72% /    # 注意这里只有13G了
/dev/sda1               1014M  168M  847M  17% /boot
tmpfs                    1.6G     0  1.6G   0% /run/user/0

查看未分配的存储

[root@master ~]# lsblk -l
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  200G  0 disk  #还有150G未分配
sda1          8:1    0    1G  0 part /boot
sda2          8:2    0   49G  0 part 
centos-root 253:0    0   44G  0 lvm  /
centos-swap 253:1    0    5G  0 lvm  
sr0          11:0    1 1024M  0 rom  
[root@master ~]# fdisk -l
Disk /dev/sda: 214.7 GB, 214748364800 bytes, 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00039057
Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   104857599    51379200   8e  Linux LVM

对新增加的磁盘进行分区

[root@master ~]# fdisk /dev/sda 
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
# 1
Command (m for help): n
Partition type:
   p   primary (2 primary, 0 extended, 2 free)
   e   extended
# 2
Select (default p): p
Partition number (3,4, default 3): 
First sector (104857600-419430399, default 104857600): 
Using default value 104857600
Last sector, +sectors or +size{K,M,G} (104857600-419430399, default 419430399): 
Using default value 419430399
Partition 3 of type Linux and of size 150 GiB is set
# 3
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

分区完后重启
分区完之后无法通过lsbfk等命令查看到,但是fdisk -l可以看到

[root@master ~]# fdisk -l
Disk /dev/sda: 214.7 GB, 214748364800 bytes, 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00039057
Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   104857599    51379200   8e  Linux LVM
/dev/sda3       104857600   419430399   157286400   83  Linux

创建物理卷

[root@master ~]# pvcreate /dev/sda3
  Physical volume "/dev/sda3" successfully created.

查看物理卷信息

[root@master ~]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               centos
  PV Size               <49.00 GiB / not usable 3.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              12543
  Free PE               0
  Allocated PE          12543
  PV UUID               1BfiJn-wrKX-mDQa-Z43E-l3XT-KEki-A9tgAj
  "/dev/sda3" is a new physical volume of "150.00 GiB"
  --- NEW Physical volume ---
  PV Name               /dev/sda3
  VG Name               # 这里是空的
  PV Size               150.00 GiB
  Allocatable           NO
  PE Size               0   
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               AKLFX6-6vhY-t38N-0FjV-r0WR-NXFG-3fafid
[root@master ~]# vgdisplay
  --- Volume group ---
  VG Name               centos #这里是系统盘的VG名
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <49.00 GiB
  PE Size               4.00 MiB
  Total PE              12543
  Alloc PE / Size       12543 / <49.00 GiB
  Free  PE / Size       0 / 0   
  VG UUID               TWyVcI-YK3A-SAhs-DQWd-0C6n-1PAZ-KumTJW

将新增加的分区/dev/sda3加入到根目录分区centos

[root@master ~]# vgextend centos /dev/sda3
  Volume group "centos" successfully extended

重新查看卷组信息

[root@master ~]# vgdisplay
  --- Volume group ---
  VG Name               centos
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               1
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               198.99 GiB
  PE Size               4.00 MiB
  Total PE              50942
  Alloc PE / Size       12543 / <49.00 GiB
  Free  PE / Size       38399 / <150.00 GiB #还有150G未分配
  VG UUID               TWyVcI-YK3A-SAhs-DQWd-0C6n-1PAZ-KumTJW

进行卷扩容

[root@master ~]# lvextend -l +100%FREE /dev/mapper/centos-root
  Size of logical volume centos/root changed from <44.00 GiB (11263 extents) to 193.99 GiB (49662 extents).
  Logical volume centos/root successfully resized.

调整卷分区大小

[root@master ~]# xfs_growfs /dev/mapper/centos-root
meta-data=/dev/mapper/centos-root isize=512    agcount=4, agsize=2883328 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=11533312, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=5631, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 11533312 to 50853888

最后再查看系统盘大小

[root@master ~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 7.8G     0  7.8G   0% /dev
tmpfs                    7.8G     0  7.8G   0% /dev/shm
tmpfs                    7.8G  5.0M  7.8G   1% /run
tmpfs                    7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/mapper/centos-root  194G   26G  169G  13% / #已经完成扩容
/dev/sda1               1014M  168M  847M  17% /boot
tmpfs                    1.6G     0  1.6G   0% /run/user/0

mysql全量备份和增量备份

发表于 2021-12-07 分类于 mysql

全量备份

备份

在MySQL中提供了命令行导出数据库数据以及文件的一种方便的工具mysqldump,

-- 备份全部数据库的数据和结构
-- -d 备份全部数据库的结构
-- -t 备份全部数据库的数据
mysqldump -uroot -p123456 -A > /data/mysqlDump/mydb.sql

-- 备份单个或多个数据库的数据和结构
mysqldump -u root -p --databases db1 db2 > xxx.sql

还原

-- 方法一：在系统命令行中
mysql -uroot -p123456 < /data/mysqlDump/mydb.sql
-- 方法二：登录进入mysql系统中
mysql> source /data/mysqlDump/mydb.sql

增量备份

要做增量备份首先要开启log_bin

-- 查看log_bin开启状态
show variables like '%log_bin%';
-- 开启后可以查看当前使用的日志文件
show master status;
-- 刷新使用的日志文件,如当前使用mysql-bin.00005,执行命令后变为mysql-bin.00006
mysqladmin -uroot -p123456 flush-logs
-- 保存mysql-bin日志文件
-- 从mysql-bin恢复数据
mysqlbinlog /var/lib/mysql/mysql-bin.000015 | mysql -uroot -p123456 db1;

脚本

#!/bin/sh
# 备份当日的
docker exec -it mysql mysqldump -uroot -p123456 -A > /root/mysql/bak/$(date +%F).sql
# 清除一周前的备份
rm -rf /root/mysql/bak/$(date -d "-1 week " +%Y-%m-%d).sql

注意：如果使用crontab去定时执行上面的脚本,需要把-it去掉,不然生成的sql文件为空.

Elasticsearch查询

发表于 2021-11-02 分类于 Elasticsearch

空查询

// 查询所有
GET /index/_search
{}
// 分页查询
GET /index/_search
{
  "from": 1,
  "size": 2
}

bool联合查询

must: 文档必须完全匹配条件
should: should下面会带一个以上的条件，至少满足一个条件
must_not: 文档必须不匹配条件

GET /index/_search
{
  "query": {
    "bool": {
      "must_not": [
        {"match": {
          "content": "中国"
        }},
        {
          "match": {
            "content": "平均"
          }
        }
      ]
    }
  }
}

高级检索

// 正则检索
GET /index/_search
{
  "query": {
    "regexp": {
      "FIELD": "REGEXP"
    }
  }
}
// 前缀检索
GET /index/_search
{
  "query": {
    "prefix": {
      "content": {
        "value": "美国"
      }
    }
  }
}
// 不分词检索
GET /index/_search
{
  "query": {
    "term": {
      "content": {
        "value": "美国"
      }
    }
  }
}
// 通配符检索
GET /index/_search
{
  "query": {
    "wildcard": {
      "content": {
        "value": "*天*"
      }
    }
  }
}
// 过滤filter
// 模糊fuzzy
// 权重

Elasticsearch中文分词器

发表于 2021-11-02 分类于 Elasticsearch

在线安装

# v6.3.0版本改成es对应的版本
./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.5.2/elasticsearch-analysis-ik-7.5.2.zip

离线安装

cd your-es-root/plugins/ && mkdir ik
# 解压到目录
your-es-root/plugins/ik
# docker安装的可以本地解压后拷贝到容器
docker cp ik/ es:/usr/share/elasticsearch/plugins/

所有ES节点都需要安装!

验证

Kibana打开Dev Tools:

测试分词

// ik_max_word: 会将文本做最细粒度的拆分
GET _analyze
{
  "text": "中国早日收复台湾","tokenizer": "ik_max_word"
}
// ik_smart: 会做最粗粒度的拆分
GET _analyze
{
  "text": "中国早日收复台湾","tokenizer": "ik_smart"
}

读写数据

// 新建index
PUT /index
// 新建type,如果ik分词器没有安装成功会报错
POST index/_mapping
{
  "properties": {
      "content": {
          "type": "text",
          "analyzer": "ik_max_word",
          "search_analyzer": "ik_smart"
      }
  }
}
// 写入数据1
POST index/_create/1
{
  "content":"美国留给伊拉克的是个烂摊子吗"
}
// 写入数据2
POST index/_create/2
{
  "content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"
}
// 写入数据3
POST index/_create/3
{
  "content":"公安部：各地校车将享最高路权"
}
// 写入数据4
POST index/_create/4
{
  "content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
}
// 查询
POST index/_search
{
  "query": {
    "match": {
      "content": "中国"
    }
  }
}

elasticsearch-analysis-ik

Elasticsearch安全验证

发表于 2021-11-02 分类于 Elasticsearch

1.创建证书

# 启动一个单机的es
# 进入容器执行下面命令
bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""
# 证书拷出容器
docker cp es:/usr/share/elasticsearch/config/elastic-certificates.p12 .
# 将证书拷贝到各个节点

2.各个节点准备es配置文件

# 集群名称保持一致
cluster.name: elasticsearch-cluster
# 集群内唯一
node.name: es-node1
network.bind_host: 0.0.0.0
# 本节点IP
network.publish_host: 192.168.3.17
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true
node.data: true
# 这些节点争抢master
cluster.initial_master_nodes: 192.168.3.17,192.168.3.19,192.168.3.20
discovery.seed_hosts: 192.168.3.19,192.168.3.20
# 安全验证相关的
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/elastic-certificates.p12

3.准备data目录

# 用于保存es数据,需要是777权限
mkdir data
chmod 777 -R data/

4.启动

docker run -d --name es --net host \
-v /root/es/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
-v /root/es/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12 \
-v /root/es/data:/usr/share/elasticsearch/data \
elasticsearch:7.5.2

5.创建用户

# 登录一台es容器内
# 自动生成好默认用户和密码
bin/elasticsearch-setup-passwords auto
# 手动输入密码
[root@data1 bin]# elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

6.验证

# 用户密码都是elastic
curl --user elastic:elastic 'localhost:9200/_cluster/health?pretty'
curl -X GET --user elastic:elastic  "localhost:9200/_cat/nodes?v&pretty"

curl中加入安全验证可以使用--user elastic:elastic,如果是postman、yapi等工具中想要加上验证需要先将elastic:elastic进行base64加密:ZWxhc3RpYzplbGFzdGlj,
构造一个字符串形如:Basic ZWxhc3RpYzplbGFzdGlj,然后添加到header中Authorization作为键,该字符串为值.

安装kibana

kibana配置

server.name: kibana
server.host: "0"
elasticsearch.hosts: ["http://192.168.3.17:9200","http://192.168.3.19:9200","http://192.168.3.20:9200"]
xpack.monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.username: "elastic"
elasticsearch.password: "elastic"
# 中文页面
i18n.locale: zh-CN

启动

docker run -d --name kibana -p 5601:5601 -v /root/es/kibana.yml:/usr/share/kibana/config/kibana.yml kibana:7.5.2

Elasticsearch-kibana部署

发表于 2021-11-02 分类于 Elasticsearch

docker run -d --name kibana -p 5601:5601 \
-e ELASTICSEARCH_HOSTS='["http://192.168.3.17:9200","http://192.168.3.19:9200","http://192.168.3.20:9200"]' \
kibana:7.5.2