Happywzy

说明

本文通过安装 docker loki plugin 直接采集docker容器日志，并推送至loki。官方文档

插件安装

# 安装最新的插件
docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
# 查看插件
[root@data1 ~]# docker plugin ls
ID             NAME          DESCRIPTION           ENABLED
744b79d5d1a9   loki:latest   Loki Logging Driver   true

插件升级

# 停用
docker plugin disable loki --force
# 升级
docker plugin upgrade loki grafana/loki-docker-driver:latest --grant-all-permissions
# 启用
docker plugin enable loki
# 重启docker
systemctl restart docker

插件卸载

docker plugin disable loki --force
docker plugin rm loki

使用

单独为一个容器设置日志驱动

# --log-driver=loki
docker run --log-driver=loki \
    --log-opt loki-url="http://IP:3100/loki/api/v1/push" \
    --log-opt loki-retries=5 \
    --log-opt loki-batch-size=400 \
    --log-opt max-size=50m \
    --log-opt max-file=3 \
    grafana/grafana

为所有的容器设置默认参数

编辑/etc/docker/daemon.json文件(如果没有就新建).

{
    "debug" : true,
    "log-driver": "loki",
    "log-opts": {
        "loki-url": "http://IP:3100/loki/api/v1/push",
        "loki-batch-size": "400",
        "loki-retries": 5,
        "max-size": "50m",
        "max-file": "10"
    }
}

更多如docker-compose的用法参考官网文档.

• loki.yaml

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9095

ingester:
  chunk_idle_period: 3m
  chunk_block_size: 262144
  chunk_retain_period: 1m
  max_transfer_retries: 0
  lifecycler:
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1

limits_config:
  reject_old_samples: true
  reject_old_samples_max_age: 168h

common:
  path_prefix: /tmp/loki
  storage:
    filesystem:
      chunks_directory: /tmp/loki/chunks
      rules_directory: /tmp/loki/rules
  replication_factor: 1
  ring:
    instance_addr: 127.0.0.1
    kvstore:
      store: inmemory

storage_config:
  boltdb_shipper:
    active_index_directory: /tmp/loki/boltdb-shipper-active
    cache_location: /tmp/loki/boltdb-shipper-cache
    cache_ttl: 24h
    shared_store: filesystem
  filesystem:
    directory: /tmp/loki/chunks

chunk_store_config:
  max_look_back_period: 672h

table_manager:
  retention_deletes_enabled: true
  retention_period: 672h

compactor:
  working_directory: /tmp/loki/boltdb-shipper-compactor
  shared_store: filesystem
  retention_enabled: true
  retention_delete_delay: 2h

query_range:
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        max_size_mb: 200

querier:
  query_timeout: 2m

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

# ruler:
#   alertmanager_url: http://localhost:9093

# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/usagestats/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
analytics:
 reporting_enabled: false

说明

本文通过 自定义注解 + Redis 实现接口限流。

实现

• 自定义注解

@Inherited
@Documented
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
// 默认就是60秒内可以访问2次
public @interface RequestLimit {
    /**
     * 时间，秒为单位
     */
    int second() default 60;
    /**
     * second时间内允许访问的次数
     */
    int maxCount() default 2;
}

• 通过AOP实现限流

@Configuration
@EnableAspectJAutoProxy
@Aspect
@Slf4j
@Order(-5) // 如果有多个AOP，需要将Order设置到最小，优先判断限流
public class RequestLimitAspect {
    // 这里通过将访问信息保存在redis中，判断是否限流
    @Autowired
    private RedisTemplate<String, String> redisTemplate;
    // 定义切点，所有添加RequestLimit注解的
    @Pointcut(value = "@annotation(org.xxx.xxx.annotation.RequestLimit)")
    private void handleRequestLimit() {
    }

    @Around("handleRequestLimit()")
    public Object handleResponseEncrypt(ProceedingJoinPoint pjp) throws Throwable {
        MethodSignature signature = (MethodSignature) pjp.getSignature();
        Method method = signature.getMethod();
        RequestLimit requestLimit = method.getAnnotation(RequestLimit.class);
        // 没有添加注解直接放行
        if (null == requestLimit) {
            return pjp.proceed();
        }
        ServletRequestAttributes requestAttributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
        // 获取HttpServletRequest
        HttpServletRequest request = (HttpServletRequest) requestAttributes.resolveReference(RequestAttributes.REFERENCE_REQUEST);
        // 获取公网IP
        String ip = CommonUtils.getPublicIp(request);
        // 实现同一个公网IP 限流
        String key = "limit:" + ip + ":" + request.getServletPath();
        String count = redisTemplate.opsForValue().get(key);
        if (StringUtils.isEmpty(count)) {
            count = "1";
            redisTemplate.opsForValue().set(key, count, requestLimit.second(), TimeUnit.SECONDS);
            return pjp.proceed();
        }
        if (Integer.valueOf(count) < requestLimit.maxCount()) {
            redisTemplate.opsForValue().increment(key);
            return pjp.proceed();
        }
        log.info("RemoteAddr:{} 请求接口:{} 请求过于频繁, 已拒绝请求.", ip, request.getServletPath());
        HttpServletResponse response = requestAttributes.getResponse();
        response.setCharacterEncoding("UTF-8");
        response.setContentType("application/json; charset=utf-8");
        response.getWriter()
                // 触发限流自定义返回内容
                .println(JSON.toJSONString(new Response(Constant.ResponseCode.RequestLimit, Constant.ResponseMsg.RequestLimit)));
        return null;

    }
}

• 获取公网IP

public static String getPublicIp(HttpServletRequest request) {
    String ip = null;
    try {
        ip = request.getHeader("x-forwarded-for");
        if (StringUtils.isEmpty(ip) || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("Proxy-Client-IP");
        }
        if (StringUtils.isEmpty(ip) || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("WL-Proxy-Client-IP");
        }
        if (StringUtils.isEmpty(ip) || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_CLIENT_IP");
        }
        if (StringUtils.isEmpty(ip) || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_X_FORWARDED_FOR");
        }
        if (StringUtils.isEmpty(ip) || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getRemoteAddr();
        }
    } catch (Exception e) {
        log.error("getPublicIp ERROR ", e);
    }
    //使用代理，则获取第一个IP地址
    if (StringUtils.isEmpty(ip) && ip.length() > 15) {
        if (ip.indexOf(",") > 0) {
            ip = ip.substring(0, ip.indexOf(","));
        }
    }
    return ip;
}

说明

springboot配置ExceptionHandler不仅可以统一处理全局异常，还可以用来自定义404、405返回。

自定义404、405

• 配置application.yml

# 捕获404异常需要开启以下配置,其它异常无需开启
spring:
  mvc:
    throw-exception-if-no-handler-found: true
  resources:
    add-mappings: false

• 编写java

@RestControllerAdvice
@Slf4j
public class ExceptionHandle {
    @ExceptionHandler(Exception.class)
    public Object handlerException(Exception e) {
        // 请求接口地址不存在 404
        if (e instanceof NoHandlerFoundException) {
            // Response是自定义的一个返回对象
            return new Response(Constant.NoHandlerFound, e.getMessage());
        }
        // 请求方法不支持 405
        if (e instanceof HttpRequestMethodNotSupportedException) {
            return new Response(Constant.MethodNotSupported, e.getMessage());
        }
        //其他异常都可以捕获
        return new Response(Constant.sysError, Constant.sysError);
    }
}

说明

gson 实现 yyyyMMddHHmmss 格式的时间与 LocalDateTime 类型相互转换.

// 自定义序列化
JsonSerializer<LocalDateTime> serializer =
                (localDateTime, type, jsonSerializationContext) -> new JsonPrimitive(localDateTime.format(DateTimeFormatter.ofPattern("yyyyMMddHHmmss")));
// 自定义反序列化
JsonDeserializer<LocalDateTime> deserializer =
        (jsonElement, type, jsonDeserializationContext) -> LocalDateTime.parse(jsonElement.getAsJsonPrimitive().getAsString(), DateTimeFormatter.ofPattern("yyyyMMddHHmmss"));
Gson gson = new GsonBuilder()
        .registerTypeAdapter(LocalDateTime.class, serializer)
        .registerTypeAdapter(LocalDateTime.class, deserializer).create();

// 测试 object to json
TestObject object = new TestObject();
object.setDate(LocalDateTime.now());
System.out.println(gson.toJson(object)); // {"date":"20221115141202"}
// 测试 json to object
String json = "{\"date\":\"20221115140534\"}";
System.out.println(gson.fromJson(json, TestObject.class)); // TestObject(date=2022-11-15T14:05:34)

• TestObject.java

@Data
@ToString
public class TestObject {
    private LocalDateTime date;
}

说明

Nginx Ingress Controller 支持通过 annotation 的方式配置服务器与客户端之间的双向 HTTPS 认证来保证连接的安全性。

创建自签的CA证书

openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 356 -nodes -subj '/CN=Fern Cert Authority'

创建Server端证书

# 生成Server端证书的key和请求文件
openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=foo.bar.com'
# 使用根证书签发Server端请求文件，生成Server端证书
openssl x509 -req -sha256 -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

创建Client端证书

# 生成Client端证书的请求文件
openssl req -new -newkey rsa:4096 -keyout client.key -out client.csr -nodes -subj '/CN=Fern'
# 使用根证书签发Client端请求文件，生成Client端证书
openssl x509 -req -sha256 -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

创建CA证书的Secret

# 默认default命名空间创建
kubectl create secret generic ca-secret --from-file=ca.crt=ca.crt

创建Server证书的Secret

kubectl create secret generic tls-secret --from-file=tls.crt=server.crt --from-file=tls.key=server.key

创建测试用的Nginx Ingress用例

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
    nginx.ingress.kubernetes.io/auth-tls-secret: "default/ca-secret"
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
  name: nginx-test
  namespace: default
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - backend:
          serviceName: http-svc
          servicePort: 80
        path: /
  tls:
  - hosts:
    - foo.bar.com
    secretName: tls-secret

测试

# 客户端不传证书访问
curl --cacert ./ca.crt  https://foo.bar.com
# 客户端传证书访问
curl --cacert ./ca.crt --cert ./client.crt --key ./client.key https://foo.bar.com

说明

nginx 证书双向认证是认证客户端证书是否是配置的根证书颁发的.

创建根证书

#创建根证书私钥：
openssl genrsa -out ca.key 1024
#创建根证书请求文件：
openssl req -new -out ca.csr -key ca.key
#创建根证书：
openssl x509 -req -in ca.csr -out ca.crt -signkey ca.key -CAcreateserial -days 3650

• ca.crt : 签名有效期为10年的根证书
• ca.key: 根证书私钥文件
• ca.csr: 根证书请求文件

根据根证书创建服务端证书

#生成服务器端证书私钥：
openssl genrsa -out server.key 1024
#生成服务器证书请求文件，过程和注意事项参考根证书，本节不详述：
openssl req -new -out server.csr -key server.key
#生成服务器端公钥证书
openssl x509 -req -in server.csr -out server.crt -signkey server.key -CA ca.crt -CAkey ca.key -CAcreateserial -days 3650

根据根证书创建客户端证书

#生成客户端证书秘钥：
openssl genrsa -out client.key 1024
#生成客户端证书请求文件，过程和注意事项参考根证书，本节不详述：
openssl req -new -out client.csr -key client.key
#生客户端证书
openssl x509 -req -in client.csr -out client.crt -signkey client.key -CA ca.crt -CAkey ca.key -CAcreateserial -days 3650
#生客户端p12格式证书，需要输入一个密码，比如123456
openssl pkcs12 -export -clcerts -in client.crt -inkey client.key -out client.p12

client.p12：客户端 p12 格式，这个证书文件包含客户端的公钥和私钥，主要用来给浏览器或 postman 访问使用.

nginx配置

server {
    listen       443 ssl;
    server_name  www.yourdomain.com;# 无域名可填写ip
    ssl                  on;  
    ssl_certificate      /data/sslKey/server.crt;  #server公钥证书
    ssl_certificate_key  /data/sslKey/server.key;  #server私钥
    ssl_client_certificate /data/sslKey/ca.crt;  #根证书，可以验证所有它颁发的客户端证书
    ssl_verify_client on;  #开启客户端证书验证  
    ssl_verify_depth 1; #如果客户端证书不是由根证书直接颁发,需要开启此参数

    location / {
        proxy_pass http://127.0.0.1:8080/;
    }
    
    #location / {
    #    root   html;
    #    index  index.html index.htm;
    # }
}

访问测试

• curl 带证书访问

   curl --cert ./client.crt --key ./client.key https://foo.bar.com

• postman

1. 设置 General 中先把 SSL certificate verification 关掉.
2. Certificates 中选择Add Certificates,PFX file 配置客户端 p12 文件和密码.