Friday, October 30, 2015

an interesting java architecture

At work, a guy named Scott built up a kind of neat Java backend for an MVP Proof of Concept we were doing, a bunch of RESTful APIs called "Legion". This project has some interesting ideas and best practices developed from Scott's experience and a greenfield chance to do things right, so I thought I'd share some of it here.

Some of the stuff may be old hat to people who have been more in the Java loop than I have; my last big Java projects were when Generics and Annotations were still relative new and fresh.

OVERVIEW

At the highest level, Legion's job is to provide RESTful endpoints for UIs etc to use to make and edit advertising related things like Campaigns/Flights/etc. It uses Spring for endpoint wiring and dependency injection. It is very self-contained, and Spring Boot means it doesn't need an Apache container.

Scott uses Spring's recommend terms for its layers: parallel to but not exactly the same as MVC.

The layers are:

  • CONTROLLER (confusingly, this is most similar to the MVC "View")
  • SERVICE (akin to MVC "Controller")
  • REPOSITORY (akin to MVC "Model")

We'll get more into these layers later. One important note is: it might be tempting to access stuff in the Repository Layer directly in the controller, since so often the Service layer (like in CampaignService) looks like simplistic one-liner plumbing, but this should absolutely be avoided because if the Service layer provides critical transaction functionality via the @Transactional annotation.

DEPENDENCY INJECTION

Spring is great for dependency injection, which is great for stuff like unit testing and what not, and otherwise loosely coupling your various components. The modern preference is for lots of singleton classes (vs, say, lots of static classes).

When you see a function annotated @Autowired, its arguments are managed /injected by Spring, generally at boot.( In fact, Spring can autowire private variable members, but it's considered better to keep it to the function level.)

Spring's injection model seems to have been influenced by Guice, so a few places I'll mention the equivalent name Guice uses. (Scott preferred Spring because it's larger age/scope means stuff like Jersey connectors are made for it.)

If you're writing your own class, you can tell Spring you want it to manage it via @Component (or one of the "subclasses" of @Component) but you can also use the @Bean annotation on a function (@Inject in guice) to have Spring manage a singleton instance of an arbitrary class, the one returned by that function. (In general its the return type class name (not the variable instance name) that's important to get the right class to where it is needed.) The @Bean trick requires @Configuration in the containing class. (In general, when Spring does its massive scan at bootup, it wants to scan every class, not  every function, since that would be inefficient.)

THE MAIN CLASS

Charmingly LegionApplication contains a good old fashioned public static void main() class.

There's a line commented out,
org.apache.ibatis.logging.LogFactory.useStdOutLogging();
which is super-useful to put back in when it comes time to see what mybatis/ibatis is doing against the database.

Log4J is also setup here, and then it hands off to Spring:
SpringApplication.run(LegionApplication.class, args);


THE CONTROLLER LAYER

Controller classes are annotated with "@RestController" - which means they are a @Controller which means they are a @Component, which means they are a bean and can be managed by Spring. (Guice uses @Bean).

THE SERVICE LAYER

As previously mentioned, wrapping stuff as a transaction might be the most crucial thing being done at this layer. Spring does its @Transactional magic by generating subclasses (on a properly configured IDE like IntelliJ, the subclasses will show up highlighted differently in the stack trace) "Obviously", if you put a breakpoint in the middle a Transactional call, changes will not show up in the database until the call is completed.

Transactional calls are re-entrant, and so one transactional function can call another and the transaction moves up to the outer layer, so to speak.

THE REPOSITORY LAYER

Most of the meat of the MVP was here. If we were using Hibernate, this would be a trivial layer, but in Scott's experience Hibernate didn't scale very well, and often made upgrading extremely hard. iBATIS (or it's current flavor MyBatis) seems like a better bet (personally I like that it is rather transparent and lets you see the SQL sausage being made.

MyBatis looks a bit like a templating language, with tags providing flow control around the SQL query meat.

MyBatis queries can be done via annotations, but Scott prefers the XML approach, as the annotation syntax gets wonky (also since SQL is kind of its own little discipline, I think it makes sense to have it all gathered in one area. In Legion's case, it provides an @Bean SqlSessionFactoryBean where the location of the mappers are "hardwired" in -
bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:mappers/*.xml"));
Roughly speaking, all the mapper XML is lurped together; having different query groups in individual .xml files is just  for human reading convenience.


So in repository java code, you see stuff like
session.selectOne("legion.selectCampaign", campaign.id);
which refers to
 <select id="selectCampaign" parameterType="long" resultMap="campaignResult">
in the XML.

MyBatis query bodies uses #{fieldname} style insertions of parameters. These are context dependent; for a single typed parameter, the name is essentially ignored, POJO beans use field names and Maps use keys.

MyBatis then can build parts of queries using tags like <where> and <if>. It's actually super clever so if you had a clause like
       <where>
            <if test="nameQuery != null">
                name LIKE #{nameQuery}
            </if>
            <if test="idQuery != null">
                OR id = #{idQuery}
            </if>
        </where>
If nameQuery was null the query would still build correctly without the "OR" that would otherwise mess the syntax up.
In general #{} is escaped and ${} is unescaped. (Meta-stuff like column names can't be escaped, for instance)

In theory MyBatis can return complex datatypes, but our version was getting cranky about nested objects, and so there is sometimes when the Java code handles additional glomming of stuff.

Another note was our MyBatis config did stuff like
<setting name="mapUnderscoreToCamelCase" value="true"/>
to hand external_id = externalId, that kind of thing.

Sometimes the return value (set as resultMap) referred to campaignResult, which was defined earlier in the file. It helped juggle the campaign type and status foreign keys  so that the Java code could do a lookup - for certain long lived data (e.g. a list of countries: content that rarely changes) Scott wanted to avoid always the expenses of doing joins and of shipping extra data over the wire, so he made a LKPCachedRepository (LKP = lookup) that will keep an in-memory lookup table.

EXCEPTIONS

One cross-layer thing we looked at was exception; with the use of the @ResponseStatus annotation on "NoSuchThingException", the repository layer could throw an exception that indicated what kind of HTTP response code message should be sent at the the controller layer. In general, the plan would be to see if we could get the UI to make sense of the code and payload, and only fiddle further if necessary, since the @ResponseStatus defaults might very well be the "right thing" in this case.

SPRING SETUP STUFF

So back to LegionApplication - we see 3 critical annotations:
@SpringBootApplication(exclude = DataSourceAutoConfiguration.class) //lets us specify our own damn datasource
@EnableTransactionManagement //lets us use @Transactional to great effect
@Import({SecurityConfig.class}) //turns on security

The @SpringBootApplication is of course implying @Configuration (i.e. having Spring manage the Beans), and @ComponentScan, where it can look for @Component annotations on all classes in the project.)

I would say looking at this helped me get my own head wrapped around Annotations; they're kind of funky in how they are almost like bits of source code that get preserved in the generated byte code to provide instructions that can be done at boot time.

application.properties includes some values like spring.datasource.username... these are injected into LegionApplication via @Value (this would be @Named in Guice) - e.g.
@Value("${spring.datasource.username}")

perhaps the most amazing one Scott made was ${use.embedded.mysql} - if true, Legion will make its own internal SQL server, running create.sql and create-static.sql - this is a technique outlined in
http://zhentao-li.blogspot.com/2013/06/using-embedded-mysql-database-for-unit.html
and is super great for unit testing. (The one downside is because of lack of support my MySql for their mxi (which is the core of this technique) the embedded mysql instance is stuck at version of 5.5.9)

TESTING

Scott made an AbstractIntegrationTest class that takes care of much of the boilerplate, so the subclasses can call the REST endpoints and check the results.
It has some cool annotations:
@RunWith(SpringJUnit4ClassRunner.class) //run with jUnit
@SpringApplicationConfiguration(classes = LegionApplication.class) //Here's the application we want to test
@WebIntegrationTest(value = {"server.port=0"}, randomPort = true) //actually boot this application on a random port, so we can run calls against it

Individual tests can describe if they clean up after themselves or if the system should do a tear down and rebuild via @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)

SECURITY

Security was fairly minimal for the MVP, but still there is SimpleCORSFilter to help deal with the cross domain issue / connection with the front end.  Even though the base class is generically "Filter" they are all HTTP and so do a lot of stuff with HttpServletResponse and headers etc.

There is also a SecurityConfig class extending WebSecurityConfigurerAdapter. Its configureGlobal() function set itself as the UserDetailsService(), and there's a shell implementation of loadUserByUsername... it is making an instance of DaoAuthenticationProvider. This class also shows us using BCrypt for some basic username stuff. (Some of the Bcrypt stuff is setup in LegionApplication)

The main configure() block looked like this, and the comments are at least as good as my current understanding:
 protected void configure(HttpSecurity http) throws Exception {
 http.csrf().disable() //disable CSRF to allow POST requests to our endpoints
.headers().httpStrictTransportSecurity().disable() //disable HSTS headers so we don't override local http servers on 8080 for a year
.authorizeRequests() //begin specifying security configurations
.antMatchers("/login").authenticated() //any user can login, if they're in our system
//ALL REAL CONFIGURATIONS MUST GO ABOVE THIS LINE
.anyRequest().permitAll() //everything else is open to everyone
.and().httpBasic(); //and use http basicAuth
}
So antMatchers was the interesting bit, and at this point in configure() you can set these wildcard style things up to have specify certain user/role requirements

That was kind of it, with the additional mention of "DHC" being a great little tool in chrome for running Ajax stuff.

No comments:

Post a Comment