Salient Soft

I was using Spring Data JPA with Oracle as part of an export process from CouchDB to Oracle. The CouchDB export application first exports my existing Places data from CouchDB, then maps it to the entity model I am using for JPA/Oracle, then persists it into Oracle, and finally performs some rudimentary listing of the Oracle data by logging it.

Some issues/points came to light during this excercise which I am noting here:-

1/ My connect string to Oracle was using a Pluggable database in Oracle XE. This needs to use a service name (xepdb1) rather than a SID (xe) for the connection. To do this I needed to use a slash before the service name in the application properties, to indicate a service name, rather than the colon I would normally use for a SID:

# Oracle settings
spring.datasource.url = jdbc:oracle:thin:@localhost:1521/xepdb1
spring.datasource.username = places
spring.datasource.password = xxxxxxxx

2/ I changed the Hibernate naming strategy in the application properties to prevent unwanted snake casing of Oracle column names:

# Hibernate settings
spring.jpa.hibernate.naming.physical-strategy = org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl

3/ For persisting to Oracle, I had the choice of using Spring Data JPA, calling save() or saveAll() on the Individual Repository interfaces that extended CrudRepository (such as PlacesRepository), or I could use the Entity Manager. The Spring Data methods would persist or update fine, and could also save an iterable in one call. However, I had some small additional logic as I was passing and returning a Map. As I chose not to put this logic in the service layer, I therefore used an ordinary JPA Dao (repository) class, and autowired the entity manager via the @PersistenceContext annotation. I could therefore incorporate my logic in a single call and use entityManager.merge() to persist. I felt that situations vary and that it was not always appropriate to use Spring Data Interfaces for some use cases. However the Spring Data find queries were useful and clever for querying the database as they allowed for example a find by key with a like, and order by, just by declaring the appropriate method calls in the interface. I also used the @Query annotation for a custom query, which again was just annotated on a method in the interface.

4/ I had issues whereby the foreign keys on dependant entities were not being auto populated and threw up as null, even when I set nullable=false on the @JoinColumn annotations. In the end it was a simple case of not populating the objects in my model on both sides of the join before persisting – i.e. I was populating a list in a parent object for a one-to-many, but not populating the parent in the dependant object. Doing this resolved the problem and everything persisted automatically with a single parent persist/merge, with hibernate allocating primary keys via Oracle sequences, and then auto populating them in the foreign keys. This stack overflow post here pointed me in the right direction on this. I kept the nullable=false annotations where appropriate to correctly match the relationship I was using, and what was in Oracle.

5/ Once I was at the point of listing everything I had persisted, using Spring Data queries and some simple logging, I was initially hit with Hibernate multiple bag fetch exceptions when listing the data. At this stage I had annotated my repository methods as @Transactional rather than the service layer, and was therefore using explicit eager loading annotations as I needed everything to load before the methods returned. This post here details the issue, and it appears that Hibernate can only eagerly load one collection at a time. My solution to this was to take the more normal route of annotating my service methods as @Transactional rather than the repository, which then allowed a number of repository calls to be made from a single service method, all in the same transaction. Whilst this did expose the service layer to the transaction architecture and javax.transaction package, it is the more normal approach and gives flexibility in the service layer. In my case I could then just revert to the default lazy loading and perform the logging in the same service layer method (which I was happy to do as this was a very basic example just to demonstrate fetching the imported data from Oracle). Everything then worked fine.

Comments Off on Using Spring Data JPA with Oracle

I just installed the latest 21c version of XE, and having created the system user, I connected to this via SQL developer, using XE as the SID. All fine so far.

When I then went to create a new schema via the CREATE USER command, I received ORA-65096: invalid common user or role name. Upon researching this I found that Oracle now uses a hierarchy of pluggable databases or PDBs inside a container database or CDB. Schemas aka users can then sit inside a PDB as normal. Inside the CDB, where I was logged in, the rules are different and common users are required which have their own rules and naming conventions.

I just wanted a simple local develpment solution for JPA use, so did not especially want to learn all the ins and outs of this. I found that an XE installation creates a default PDB called XEPDB1, and managed to create an SQL developer connection to this using the already created system account. I tried using XEPDB1 as the SID rather than XE, but this failed as invalid. However when I tried XEPDB1 as a service name rather than a SID it all worked fine. I did not need to make any other changes to the connection settings – user (system), password, and port (1521) were identical.

Having then connected via this new connection, I was successfully able to create a new schema in the default PDB. As expected, to connect to this new schema I again had to use the service name of XEPDB1 and I was able to connect, once I had granted the CREATE SESSION privilege to the new user after creating it to allow me to log on. I could then proceed to grant the necessary privileges to the user and create/populate tables etc. Initially, in addition to granting CREATE SESSION, I just simply granted UNLIMITED TABLESPACE to allow the schema user to allow tables to be populated. With this I was successfully able to perform inserts, upates, deletes, and selects on all the tables present.

Comments Off on Oracle XE 21c – Container vs Pluggable Databases – CDBs vs PDBs

I had a legacy EA 12 model for the places guide database which I had used as the basis for the CouchDB implementation.

I wanted to explore a relational database as an alternative route, with the aim of not bothering with completely offline access due to the devlopment effort needed, and the lessening need due to improvements in mobile data access in the last few years. Initially I created an Oracle version of the model file, and tweaked the PK and FK  data types from UUID to long and checked a few other things.

I then tried selecting the domain model in the explorer, and using Package/Database Engineering/Generate Package DDL from the toolbar. This brought up the relevant dialogue to do the export, but try as I might it could not find any entities to export as DDL.

I then found this post here, which points out that you need to transform the relevant entities to DDL first to enable this to work. I did this using Package/Model Transformation (MDA)/Transform Current Package. You then select the entities to transform and select DDL on the right under transformations, and then hit Do Transform. This creates a DDL diagram. The original Generate Package DDL option above then worked – I was able to find all the desired entities and transform them. I had to visit each entity to select the target database DDL format, in my case Oracle, or else I got an error, but once this was done, it all worked fine.

I elected to create a single DDL SQL file and did not worry about the entity ordering – you can reorder when doing the DDL generation but I did not bother. The output DDL was pretty good as a starter and was clear. It also appeared to have been created in a way that was order independent – the tables were all created first, followed by all the foreign key contraints, so that all the tables existed prior to creating the constraints, making it order independent.

Some tweaking of the model will be needed, especially in the area of Place Features related tables, and also sequence generation, as I would likely use Oracle sequences for PK generation, via JPA/Hibernate. However, I had reached a good trial starting point which avoided the need to create all the DDL manually from scratch.

Comments Off on Exporting and Enterprise Architect 12 model as SQL/DDL statements

I decided to go from Java 8 to the most recent LTS release of Java, JDK 17, and to the current latest Spring Boot, 2.6.7, for my Spring Boot Identity API which is a java back end interface to Yoti.

Initially I used Oracle JDK 17, and I hit an issue when running live with Yoti, but using the Java IdentityAPI locally. Curiously (and annoyingly) everything worked fine when running in debug under the latest eclipse, 2022-03, but I failed to get a Yoti profile back when running the compiled Spring Boot version directly.

This Spring Boot issue details the problem, which is exactly what I was getting – an error returned when getting the Yoti profile, due to an interaction between Spring Boot, Oracle jdk 17, and Bouncy Castle encryption, causing an IllegalStateException citing ‘zip file closed’ when the jar is being unpacked. The issue does not occur when running an exploded deployment (i.e. not in a fat jar), such as is done when running in debug in eclipse as I was.

There is no resolution at present, but as the problem does not occur under openJDK, I elected to switch to openJDK 17 rather than revert to Oracle JDK 11, which would have been another option.

Switching to openJDK 17 fixed the problem, and I was able to successfully perform age verification using a local Spring Boot IdentityAPI of mine, built with openJDK 17, and using the production Yoti to retrieve a live profile.

Whilst doing this, I improved the logging and elected to use Lombok’s @slf4j annotation to create loggers, for convenience, as this also allows logging to be tweaked via the Spring Boot application.properties as normal for Spring Boot. This is detailed here for reference.

In addition I upgraded the Yoti Java SDK I was using to the latest one (yoti-sdk-api version 3.4.0) rather than the old one I was using (yoti-sdk-impl version 2.6.0). This involved a small change to my YotiService as the interface to Yoti has changed, and some small pom changes for the new version. This all worked fine.

Comments Off on Issues upgrading my Yoti Java IdentityAPI to JDK 17

I had to do this again for a new Windows 10 installation and had forgotten all the details, so am recording it here to save searching it out next time. Actually the help on Bitbucket covered it well.

1/ My preferred way to authenticate with Bitbucket is to use an ssh keypair, which is set up with ssh-keygen

2/ Follow the instructions on bitbucket here. In my case, I used the defaults for everything including the file location, and did not use a pass phrase. Per the instructions, as I was using Windows 10, ssh-keygen ran fine direct from a command prompt with no issues, rather than needing a git bash prompt.

3/ As per the instructions, I added the public key to bitbucket.

4/ I added the SSH configuration to Webstorm under File/settings/Tools/SSH Configurations. The host was bitbucket.org with the default port. The user was the bitbucket username. I picked key pair as the authentication method, and webstorm already had the correct default location for the ssh private key file. I left the passphrase empty, and hit test connection – this worked ok, so I saved the settings.

5/ When I did my first commit/push, webstorm requested a git user and email so that the change would be tagged correctly – note that this is not an authentication username, so I just used my standard abbreviation that I use with source control.

Following this, commit and push worked fine.

Comments Off on Adding Bitbucket Authentication to Webstorm

I needed to use ssl for a Spring Boot backend web service for Yoti, as https is a Yoti requirement.

This post here details how to do this for Spring Boot. The official Oracle docs can also be found here.

The following configuration fragment from the Spring Boot application.properties shows how to configure for ssl using the created keystore (keystore.p12), which was placed in the deployment directory alongside the spring boot jar:-

server.port=8443
server.ssl.key-store-type=PKCS12
server.ssl.key-store=salientsoft.p12
server.ssl.key-store-password=<password>
server.ssl.key-alias=salientsoftdev

Comments Off on Using self signed SSL with Spring Boot

I noticed this when a web service returned a blank reason phrase, which tripped up Angular due to an Angular issue which I have documented here.

It appears to be a Spring Boot Tomcat version change issue – a later Tomcat ignores the reason phrase and does not return it, as detailed here.

As reason phrases are being deprecated anyway, and I don’t use them, I have just ignored the issue, but it is worth being aware of it when debugging.

Comments Off on Spring Boot no longer returns HTTP reason phrase in responses

I was using this to add CORS headers dynamically based on application.properties.

However initially Spring stubbornly refused to add the headers.

My initial (incorrect) code was as follows:-

@Configuration
@EnableWebMvc
public class WebConfig implements WebMvcConfigurer {
 
    @Autowired
    private ConfigProperties config;
   
    @Override
    public void addCorsMappings(CorsRegistry registry) {
        config.getCorsOriginsAllowed().forEach(origin -> registry.addMapping(origin).allowedOrigins(origin));
    }
}

The error was that I had used origin both as the registry mapping key and the resulting allowed origin.

The correct code should have used a URL path as the mapping key. The correct replacement line was as follows:-

        config.getCorsOriginsAllowed().forEach(origin -> registry.addMapping("/**").allowedOrigins(origin));

This fixed version mapped every URL path to the given set of origins, using “/**” as above.

This fixed the problem.

This post here by Baeldung and this Spring guide post detail the various ways to add CORS headers, both in a fixed static way using annotations, and in dynamic ways such as the above.

Comments Off on Spring Boot not adding CORS headers in response when using WebMvcConfigurer

STS tries to warn you when editing application.properties, if any properties do not match your configuration classes.

Unfortunately it sometimes gets out of step and does not give correct warnings.

I had a few intermittent instances of this which were cleared by refreshing/rebuilding/reloading.

However one persistent one was that whilst Spring Boot was happy with the map[key] syntax for specifying a property for a map entry, the IDE continued to give a warning on this.

In the end I could not resolve this, but using the map.key syntax instead was fine and the IDE was happy with this and did not give warnings.

The only real need for the [] syntax would be to support keys with special characters in like “.”, but in my case this was not an issue and I would normally avoid this anyway if at all possible.

Comments Off on Spring Tools Suite shows inconsistent warnings re applications.properties entries.

I was using @Configuration on a configuration class in order to load its properties from application.properties.

This is detailed in this post here.

As per the post, Spring Boot supports lists and maps in the way you would expect, and handles nested objects including Collections such as maps and lists.

It can therefore handle a complete object graph of configuration properties in a clean manner, and also happily supports generics too.

Map keys can be specified either using the “.key” syntax, or the bracketed [key] syntax (which is also used with numeric values to create lists).

The problem I hit was that a map property group, used to create mock response objects keyed on a token string, was being completely ignored even though correctly specified in application.properties.

There were no errors or messages about this the log, even when debug logging was turned on – Spring appeared to be just ignoring the properties completely.

After some significant head banging the problem was trivial.

I was using Lombok to eliminate boiler plate, and both the map class and its descendants  had Lombok @AllArgsContructors and static factory methods for my own object creation.

However this meant that they did not have a no args constructor as I had not specified one. I did not need one, but Spring Boot absolutely relies on this to work!

Once I created the required constructor via @NoArgsConstructor, everything worked fine.

Hats off to Spring for this excellent way to manage config as a nice object graph, but it would be useful to have some kind of warning when a target class does not have a no args constructor!

Comments Off on Spring Boot @Configuration class ignoring some application.properties entries