好得很程序员自学网

<tfoot draggable='sEl'></tfoot>

多数据源模式JPA整合sharding-jdbc实现数据脱敏

 

前言

前一篇博文, 透明化Sharding-JDBC数据库字段加解密方案

已经完整的介绍了数据库脱敏的场景以及方案,来自京东数科的Sharding-JDBC开源项目通过对数据源中间代理的方式透明化的实现了这个功能,但是,功能虽然实现了,sql兼容的小问题还是很多,比如目前还不支持子查询,数据库定义的关键字不允许使用,等等问题,反观我们需要加解密的字段,其实占比非常小,即使遇到了和组件不兼容的地方也可以稍加改动解决掉,所以最后博主给出了一个比较完善的组件集成方案:多数据源模式,需要加解密的数据源和业务其他数据源隔离。

既解决了数据库字段加解密的额问题,同时也解决了组件对sql的兼容问题。

下面是具体的集成步骤以及需要注意的点

 

引入依赖

?

1

2

3

4

5

< dependency >

     < groupId >org.apache.shardingsphere</ groupId >

     < artifactId >sharding-jdbc-spring-boot-starter</ artifactId >

     < version >${sharding.jdbc.version}</ version >

</ dependency >

这里需要说明下,虽然采用多数据源兼容后,不能使用组件基于spring boot自动装配功能了,但是这里还是建议导入sharding-spring-boot-starter包,因为这个包下内置了配置映射的类,在自定义数据源的时候非常有用

 

添加sharding数据源配置

?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

#数据库源配置

spring.shardingsphere.datasource.name = ds

spring.shardingsphere.datasource.ds.type = com.zaxxer.hikari.HikariDataSource

spring.shardingsphere.datasource.ds.driver- class -name = com.mysql.jdbc.Driver

spring.shardingsphere.datasource.ds.jdbc-url = jdbc:mysql: //xxx?autoReconnect=true&useUnicode=true&characterEncoding=utf-8

spring.shardingsphere.datasource.ds.username = root

spring.shardingsphere.datasource.ds.password = xxx

spring.shardingsphere.encrypt.encryptors.encryptor_aes.type = aes

spring.shardingsphere.encrypt.encryptors.encryptor_aes.props.aes.key.value = 123456

spring.shardingsphere.encrypt.tables.account.columns.password.plainColumn = password

spring.shardingsphere.encrypt.tables.account.columns.password.cipherColumn = password_encrypt

spring.shardingsphere.encrypt.tables.account.columns.password.encryptor = encryptor_aes

spring.shardingsphere.props.sql.show = true

spring.shardingsphere.props.query.with.cipher.column = true

 

排除自动装配

?

1

@SpringBootApplication (exclude = SpringBootConfiguration. class )

由于导入了starter包,所以这里需要手动排除自动装载类,

 

业务数据源配置

多数据源后,业务本身的数据源也需要手动配置,默认的spring boot jpa自动转载类会判断上线文中是否存在EntityManagerFactory类,如果有就不会初始化了,所以两个数据源都需要手动配置

?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

@Configuration

@EnableConfigurationProperties (JpaProperties. class )

public class DataSourceConfiguration{

     private final JpaProperties jpaProperties;

     private final Environment environment;

 

     public DataSourceConfiguration(JpaProperties jpaProperties, Environment environment) {

         this .jpaProperties = jpaProperties;

         this .environment = environment;

     }

     @Primary

     @Bean

     public DataSource dataSource(){

         String prefix = "spring.shardingsphere.datasource." ;

         String each = getDataSourceNames(prefix).get( 0 );

         try {

             return   getDataSource(prefix, each);

         } catch ( final ReflectiveOperationException ex) {

             throw new ShardingSphereException( "Can't find datasource type!" , ex);

         }

     }

     @Primary

     @Bean

     public EntityManagerFactory entityManagerFactory() {

         HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();

         vendorAdapter.setDatabase(Database.MYSQL);

         vendorAdapter.setGenerateDdl( true );

         vendorAdapter.setShowSql( true );

         LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();

         factory.setJpaVendorAdapter(vendorAdapter);

         factory.setPersistenceUnitName( "default" );

         factory.setPackagesToScan(Constants.BASE_PACKAGES);

         factory.setDataSource(dataSource());

         factory.setJpaPropertyMap(jpaProperties.getProperties());

         factory.afterPropertiesSet();

         return factory.getObject();

     }

     @Bean

     @Primary

     public EntityManager entityManager(EntityManagerFactory entityManagerFactory){

         return SharedEntityManagerCreator.createSharedEntityManager(entityManagerFactory);

     }

     @Primary

     @Bean

     public PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory){

         JpaTransactionManager txManager = new JpaTransactionManager();

         txManager.setEntityManagerFactory(entityManagerFactory);

         return txManager;

     }

     private ListgetDataSourceNames( final String prefix) {

         StandardEnvironment standardEnv = (StandardEnvironment) environment;

         standardEnv.setIgnoreUnresolvableNestedPlaceholders( true );

         return null == standardEnv.getProperty(prefix + "name" )

                 ? new InlineExpressionParser(standardEnv.getProperty(prefix + "names" )).splitAndEvaluate() : Collections.singletonList(standardEnv.getProperty(prefix + "name" ));

     }

     @SuppressWarnings ( "unchecked" )

     private DataSource getDataSource( final String prefix, final String dataSourceName) throws ReflectiveOperationException {

         Map dataSourceProps = PropertyUtil.handle(environment, prefix + dataSourceName.trim(), Map. class );

         Preconditions.checkState(!dataSourceProps.isEmpty(), "Wrong datasource properties!" );

         DataSource result = DataSourceUtil.getDataSource(dataSourceProps.get( "type" ).toString(), dataSourceProps);

         DataSourcePropertiesSetterHolder.getDataSourcePropertiesSetterByType(dataSourceProps.get( "type" ).toString()).ifPresent(

                 dataSourcePropertiesSetter -> dataSourcePropertiesSetter.propertiesSet(environment, prefix, dataSourceName, result));

         return result;

     }

}

上面代码需要注意三个地方

一是数据源的配置,是以sharding的配置来解析获得的,是因为我们已经集成过了,不想改动配置,所以如此,如果还没集成过,可以直接使用spring 配置数据源的方式配置即可。

二是EntityManager的初始化,通过SharedEntityManagerCreator包装了下,是因为我们业务的查询通过继承SimpleJpaRepository来扩展功能的,通过SharedEntityManagerCreator包装保留了完整的事务功能。

三是需要给所有的业务数据源的配置添加 @Primary注解,让sprign上下文默认使用业务数据源

 

加解密数据源配置

?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

/**

  * @author: kl @kailing.pub

  * @date: 2020/5/18

  */

@Configuration

@EnableConfigurationProperties ({JpaProperties. class ,SpringBootEncryptRuleConfigurationProperties. class , SpringBootPropertiesConfigurationProperties. class })

@AutoConfigureAfter ({DataSourceAutoConfiguration. class , DataSourceConfiguration. class })

public class EncryptDataSourceConfiguration {

     private final SpringBootPropertiesConfigurationProperties props;

     private final SpringBootEncryptRuleConfigurationProperties encryptRule;

     private final JpaProperties jpaProperties;

     private final DataSource dataSource;

     public EncryptDataSourceConfiguration(SpringBootPropertiesConfigurationProperties props, SpringBootEncryptRuleConfigurationProperties encryptRule, JpaProperties jpaProperties, DataSource dataSource) {

         this .props = props;

         this .encryptRule = encryptRule;

         this .jpaProperties = jpaProperties;

         this .dataSource = dataSource;

     }

     @Bean

     public DataSource encryptDataSource() throws SQLException {

         return EncryptDataSourceFactory.createDataSource(dataSource, new EncryptRuleConfigurationYamlSwapper().swap(encryptRule), props.getProps());

     }

     @Bean

     public EntityManagerFactory encryptEntityManagerFactory() throws SQLException {

         HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();

         vendorAdapter.setDatabase(Database.MYSQL);

         LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();

         factory.setJpaVendorAdapter(vendorAdapter);

         factory.setPersistenceUnitName( "encryptPersistenceUnit" );

         factory.setPackagesToScan(Constants.BASE_PACKAGES);

         factory.setDataSource(encryptDataSource());

         factory.setJpaPropertyMap(jpaProperties.getProperties());

         factory.afterPropertiesSet();

         return factory.getObject();

     }

     @Bean

     public EntityManager encryptEntityManager() throws SQLException {

         return SharedEntityManagerCreator.createSharedEntityManager(encryptEntityManagerFactory());

     }

     @Bean

     public PlatformTransactionManager encryptTransactionManager() throws SQLException {

         JpaTransactionManager txManager = new JpaTransactionManager();

         txManager.setEntityManagerFactory(encryptEntityManagerFactory());

         return txManager;

     }

}

加解密数据源的源来自于业务数据源,只是在这里给业务数据源又代理了一层加解密的逻辑。加解密的规则配置采用了sharding-spring-boot-starter包中的映射类,所以可以保留和spring boot配置方式一致。

 

加解密数据源的使用

在使用时,因为默认使用的是业务数据源,所以需要在需要加解密的地方通过@Qualifier("encryptEntityManager")显示的注入加解密的数据源代理,如:

?

1

2

3

4

5

6

7

8

9

10

11

12

@Repository

public class AccountRepository extends AbstractJpaRepository {

 

     public AccountRepository( @Qualifier ( "encryptEntityManager" ) EntityManager em) {

         super (AccountModel. class , em);

     }

     @Override

     @Transactional (transactionManager = "encryptTransactionManager" )

     public   S save(S entity) {

         return super .save(entity);

     }

}

另,需要手动指定加解密数据源的事务管理器

 

结语

没有十全十美的组件,Sharding-JDBC的数据脱敏方案已经趋向于完美了。由于组件本身的架构设计,确实不好做到100%的兼容。在发现加解密组件不支持子查询时,博主发现实现这个功能很简单,尝试过向官方添加这个功能提交pr。经过对组件的进一步了解发现,从全局考虑实现这个功能非常复杂,也就放弃了。目前这个多数据源模式可以很好的解决这个问题,如果有更好的集成方案,欢迎在下面留言交流

以上就是多数据源模式JPA整合sharding-jdbc实现数据脱敏的详细内容,更多关于多数据源JPA整合sharding-jdbc数据脱敏的资料请关注其它相关文章!

原文链接:http://HdhCmsTestkailing.pub/article/index/arcid/279.html

查看更多关于多数据源模式JPA整合sharding-jdbc实现数据脱敏的详细内容...

  阅读:30次