Java

Introducing YaaS Java SDK

2017-05-28 DSP2017, Hybris, Java, Open source, RxJava, YaaS No comments

Introduction

In my company, there’s a concept of so-called “innovation day”. I have the possibility to “use” 1 innovation day per 2 development sprints. Last year, I used only 1 day due to the tight release schedule and a lot of work. Now, we are right after release, so I had time to take innovation day once again. I’ve decided to create YaaS Java SDK. If you don’t know what the YaaS is, check out my previous article about Basic usage of YaaS proxy for the microservice. In a few words, it’s a proxy for the microservices with authorization & monitoring capabilities, which allows using other services available on the YaaS market. SDK created by me is really simple, was created in a short period of time and does not cover all features of the YaaS. This SDK allows performing authorized requests to the microservices hidden behind YaaS proxy.

Tech stack used for this project is as follows:

For unit testing I used:

Quick start

I wanted to make this SDK as simple as possible so the user can add YaaS integration to the Java application within just a few lines of code.

YaaSProject project = new YaaSProject.Builder()
    .withClientId("YOUR_CLIENT_ID")
    .withClientSecret("YOUR_CLIENT_SECRET")
    .withOrganization("YOUR_ORGANIZATION")
    .withService("YOUR_SERVICE")
    .withVersion("v1")
    .withZone(Zone.EU)
    .build();

Client client = new YaaS(project);

client.get("path/to/your/endpoint")
    .subscribeOn(Schedulers.newThread())
    .blockingSubscribe(response -> System.out.println(response.body().string()));

As you can see, it looks really simple and straightforward. In the code snippet above, we’ve done the following thigs:

  1. Defined YaaS Project with YaaS service
  2. Created YaaS Client
  3. Performed HTTP GET request to the endpoint of the microservice on the new thread
  4. Received and printed body of the HTTP response from the microservice on the current thread as a String

All of that was done with Flowable type from RxJava2, which wraps Response type from OkHttp. We have a reactive stream of HTTP response here and we can do with it whatever RxJava2 offers us. Like filtering, mapping, throttling, combining it with other stream and so on.

For more information, visit repository of the project at: https://github.com/pwittchen/yaas-java-sdk.

Future plans

I have the following plans related to this project, which may be realized when I’ll have time:

  • Add more unit tests (I didn’t have enough time to cover all cases)
  • Add continuous integration
  • Integrate YaaS with SAP Hybris Backoffice or SAP Hybris Core Platform through this SDK (PoC)
  • YaaS Android SDK (copy YaaS Java SDK, downgrade it to Java 7 & optionally migrate to Kotlin and create sample mobile app)
  • Optionally, add more features to YaaS Java SDK
  • Optionally, deploy an artifact to Maven Central repository

Links

Interesting links related to this article:

Releasing Prefser v. 2.0.7

2017-05-28 Android, DSP2017, Java, Open source No comments

I’ve recently released new version of Prefser. It’s a wrapper for Android SharedPreferences with object serialization and RxJava Observables.
The new version number is 2.0.7.

In this release, I performed mostly internal work not related to the external library API. Nevertheless, it’s important for the library development in the future.

The following things were done:

  • updated dependencies
  • updated Gradle configuration
  • migrated unit tests to Robolectric
  • started executing unit tests on Travis CI
  • added integration with codecov.io and coverage report
  • extracted code related to accessors from the Prefser class (refactoring library internals)

Organizational work is done and now I’m ready for migration to RxJava2 in this project on a separate branch. I want to keep backward compatibility with RxJava1 as in my other projects. This update is planned for version 2.1.0.

Stay tuned!

Joining lists of RxJava Observables

2017-05-15 DSP2017, Java, RxJava No comments

In RxJava we have a few operators for joining Observables. The most common are:

Take a look at the documentation in these links. It has interactive marble diagrams showing how the operators work on the streams. You can move marbles along the lines and see how the output stream changes. It really helps to understand how it works.

Code snippets in this article are based on RxJava 2.1.0 with JUnit 4.12 and Google Truth 0.32 for unit tests.

Let’s say, we have the following Observables:

public Observable<String> emitNumbers() {
  return Observable.fromArray("1", "2", "3", "4").delay(1, TimeUnit.SECONDS);
}

public Observable<String> emitLetters() {
  return Observable.fromArray("a", "b", "c", "d");
}

We can merge them in the different ways.

Concat

Concat operator emits the emissions from two or more Observables without interleaving them.

We can perform the following operation:

public Observable<String> concatStreams() {
  return Observable.concat(emitNumbers(), emitLetters());
}

The easiest way to verify, how this operator works, is to create exploratory unit test as follows:

@Test
public void shouldConcatStreams() {
  // given
  Observable<String> observable = playground.concatStreams();
  List<String> expectedValues = Arrays.asList("1","2","3","4","a","b","c","d");
  List<String> joinedValues = new ArrayList<>();

  // when
  observable.blockingSubscribe(s -> joinedValues.add(s));

  // then
  assertThat(joinedValues).isEqualTo(expectedValues);
}

This operation can be represented graphically as well.

         1 --- 2 --- 3 --- 4
                  |
         a --- b --- c --- d
                  |
                  |
                concat
                  |
                 \|/
1 -- 2 -- 3 -- 4 --- a -- b -- c -- d  

As we can see one stream is appended to another regardless of the execution time of both streams.

Merge

Merge operator combines multiple Observables into one by merging their emissions.

Here we have a similar story, but changed operator:

public Observable<String> mergeStreams() {
  return Observable.merge(emitNumbers(), emitLetters());
}

We are writing another unit test:

@Test
public void shouldMergeStreams() {
  // given
  Observable<String> observable = playground.mergeStreams();
  List<String> expectedValues = Arrays.asList("a","b","c","d","1","2","3","4");
  List<String> joinedValues = new ArrayList<>();

  // when
  observable.blockingSubscribe(s -> joinedValues.add(s));

  // then
  assertThat(joinedValues).isEqualTo(expectedValues);
}

Merge operation should look like that:

         1 --- 2 --- 3 --- 4
                  |
         a --- b --- c --- d
                  |
                  |
                merge
                  |
                 \|/
a -- b -- c -- d --- 1 -- 2 -- 3 -- 4

This operator doesn’t synchronize the streams and merges them as values are emitted. Numbers are emitted later than letters, so letters are placed in the beginning of the output stream. Try to manipulate marble on the interactive diagram on the reactivex.io website to see how it should work.

Zip

The last operator, I’d like to discuss in this article is “Zip” operator. Zip combines the emissions of multiple Observables together via a specified function and emit single items for each combination based on the results of this function.

In simple words, it waits until many observables are emitted and then combines them into a pair (or triple Observable, etc. in the case or more Observables).

Now, we need to create a function, which will transform our streams and return combined stream.

public Observable<String> zipStreams() {
  return Observable.zip(emitNumbers(), emitLetters(),
      (s1, s2) -> String.format("(%s,%s)", s1, s2));
}

Next, we can verify it with test as usual:

@Test
public void shouldZipStreams() {
  // given
  Observable<String> observable = playground.zipStreams();
  List<String> expectedValues = Arrays.asList("(1,a)","(2,b)","(3,c)","(4,d)");
  List<String> joinedValues = new ArrayList<>();

  // when
  observable.blockingSubscribe(s -> joinedValues.add(s));

  // then
  assertThat(joinedValues).isEqualTo(expectedValues);
}

and it can be represented graphically like that:

        1 --- 2 --- 3 --- 4
                 |
        a --- b --- c --- d
                 |
                 |
                zip
                 |
                \|/
 (1,a) -- (2,b) --- (3,c) -- (4,d)

Now, we have pairs of merged streams.

Summary

Of course, RxJava is complicated library and these methods are not covering all possibilities of merging and combining the Observable streams. Neverhteless, examples in this article are quite basic and may help you to understand how mentioned operators work. After that we can apply the best operator to appropriate situation.


Reference thread on StackOverflow: http://stackoverflow.com/questions/28843318/android-rxjava-joining-lists

Emitting different RxJava Observables depending on the condition

2017-05-14 DSP2017, Java, RxJava No comments

Sometimes, we may need to emit different RxJava Observables depending on the specific condition dynamically. Moreover, it’s good to do it right without breaking a chain (stream of Observables). We want to combine different Observables together and do not want to nest one subscription inside another subscription because this will lead us to “subscription hell” similar to “callback hell”. Luckily RxJava has mechanisms to deal with such problems. In this article, I’m basing my examples on RxJava 2.1.0.

Let’s say we have two Observables:

public Observable<String> trueObservable() {
  return Observable.fromCallable(() -> "trueObservable");
}

public Observable<String> falseObservable() {
  return Observable.fromCallable(() -> "falseObservable");
}

and we have another Observable wrapping Boolean value:

public Observable<Boolean> createCondition(boolean returnedValue) {
  return Observable.fromCallable(() -> returnedValue);
}

This Observable can emit true or false depending on the provided parameter.

What we want to do is to:

  • emit trueObservable() when createCondition(boolean) returns true
  • emit falseObservable() when createCondition(boolean) returns false
  • emit falseObservable() when createCondition(boolean) emits empty Observable (default behaviour)

We can do it in the following way:

public Observable<String> emitTrueObservableDynamically() {
  return createCondition(true)
      .defaultIfEmpty(false)
      .flatMap(condition -> condition ? trueObservable() : falseObservable());
}

In such case, this method will emit trueObservable(). When we change parameter of the createCondition(boolean) method to false, Observable will emit falseObservable(). When we replace createCondition(boolean) method with Observable.empty(), method will return falseObservable() by default. As we can see, it’s easily solved with flatMap and defaultIfEmpty operators.

This is quite useful technique, which we can apply to reactive applications to control our flow without breaking the chain. Please note, it’s just an example you can create more complicated constructions and handle more complicated types than just boolean and more than two use cases.


Reference thread for this article on StackOverflow: http://stackoverflow.com/questions/34195218/rxjava-exequte-observable-only-if-first-was-empty.

Basic usage of YaaS as a proxy for the microservice

2017-04-30 DSP2017, Hybris, Microservices, YaaS No comments

Introduction

The company, where I currently work – SAP Hybris is developing a project called YaaS, which is an abbreviation of Hybris as a Service. Unfortunately, this article is not sponsored yet :). What a pity :(. I just like to understand many things & how they work to see the bigger picture. Moreover, company strategy is to leverage YaaS and search for the new possibilities and use cases of this project. There are situations where delegating some work to a separate service makes sense so this knowledge may be useful even when we’re developing the monolithic enterprise applications. That’s why I wrote this article. I work in a completely different project – Enterprise Commerce Platform, where I’m the part of the Backoffice team. As you can read on the official website, YaaS is a microservices ecosystem helping businesses to rapidly augment and build new, highly flexible solutions. It’s kind of marketing statement, which business people may like. Nevertheless, for developers, it’s just a bunch of buzzwords, which does not help you to understand this project. One of the aims of this article is to explain it in a simple and clear way.

yaas - hybris as a service

From the technical point of view, YaaS gives you the following possibilities:

  • it can be a proxy for your microservice, which can be deployed anywhere
  • it gives you separate proxy servers for EU and US, which you can use depending on the server or user location
  • it provides you a domain like api.eu.yaas.io/yourorganization/yourservice/
  • it provides secured connection
  • it gives a mechanism, which allows you to secure endpoints of your microservice via dynamically generated token
  • it gives you the possibility to manage access to your service for advanced use cases with features like clients, roles, etc.
  • it gives you monitoring possibilities
  • it allows you to perform versioning of your API
  • it allows you to integrate other services/packages from YaaS Market with your service
  • it gives you web interface called YaaS Builder, which you can use for managing your projects and organization

YaaS is NOT:

  • the hosting platform like Heroku or AWS – you need to have another place where you can deploy your service (like Heroku or whatever)
  • the part of the Core Hybris Platform – it’s completely separate project, but it can be integrated with the Hybris Platform

The official website of the project is: www.yaas.io.

In this article, I won’t explain all the features of YaaS. I will simply show you:

  • how to create a simple proxy for your microservice
  • how to secure endpoint of your microservice
  • how to access secured endpoint of your microservice

Maybe I’ll explain more features in the separate articles in the future.

Creating a simple proxy for the microservice

We need to do the following steps:

  1. Go to https://builder.yaas.io
  2. Create an account & log in
  3. Create an organization
  4. Create a project
  5. Within the project create a service
  6. Provide address to your service
  7. Provide API version (e.g. v1)
  8. Deploy service
  9. Right now, your service is deployed, but not accessible yet
  10. Create a Client and assign it to your service
  11. Now you should be able to access your service at: https://api.eu.yaas.io/orgranization/service/v1

Below you can see a screenshot from service configuration inside the YaaS Builder.

Securing the endpoint of the microservice

We have created our service. Now, we want to secure its endpoint. To do so, we can create Authorization Rule from the Service configuration inside YaaS Builder.

We can define methods of the HTTP request, endpoint address, and other parameters.

When we’re done, we can proceed to more tricky part. Authorization procedure of the microservice endpoint is presented in the scheme below.

calling microservice with authorization via YaaS proxy

First, we need to obtain Bearer ID. To do so, we need to perform HTTP request with Client ID and Client Secret. We can do it from terminal via curl:

curl -X POST -i 
-H "Content-Type: application/x-www-form-urlencoded" 
-d "grant_type=client_credentials&client_id=YOUR_CLIENT_ID&client_secret=YOUR_CLIENT_SECRET" 
https://api.eu.yaas.io/hybris/oauth2/v1/token

Then, we’ll receive a response like that:

{"token_type":"Bearer","access_token":"023-018f03da-cdb7-4710-a4cf-70f89e23003f","expires_in":3600,"scope":"hybris.tenant=pwtest"}

and we can make an authorized call to our microservice:

curl -X POST -i 
-H "Authorization: Bearer 023-018f03da-cdb7-4710-a4cf-70f89e23003f" 
-H "Content-Type: application/json" 
-d "" 
https://api.eu.yaas.io/pwittchen/test/v1/endpoint

after that, we should receive a response from the microservice. Note, that Bearer ID will be valid for the particular amount of time.

Hint #1: To make calls more readable in the article, I split them into lines. If you’re making a real call it’s better to have whole instruction in a single line.
Hint #2: You can get Client Secret and Client ID from the YaaS Builder while editing your Client.

Summary

As we could see, creating a microservice proxy and securing the endpoint is not so complicated, but it’s not straightforward as well. It requires some knowledge about YaaS and its design. Using this approach won’t be a good idea every time, but I think there are use cases when it can be useful. Especially, when we care about monitoring & security and when we want to have unified & controlled access to our services.

Here are a few of my ideas of delegating work to the microservice from the monolithic enterprise commerce application:

  • file or image storage
  • backups of the data
  • classification of the products – e.g. we can delegate images to the external service, which will use machine learning and neural networks to classify products by colors or by something else
  • long running operations & queues – e.g. we can delegate such things to the separate microservice to relieve CPU & Memory of the server, where core system is running and simply receive push notification with final result of the operation from the microservice when the work is done
  • sending e-mails and other types of notifications
  • and more… (if you have your own ideas – share them in comments!)

I think the basic idea could be the distribution of computations to the different servers, what may extend capabilities of the core system, make it faster, lighter and more stable. In addition, it should make work of developers easier and more joyful because they could work on the smaller parts of the system, which have a clearly specified goal and smaller codebase, which is easier to manage.

ReactiveNetwork – release v. 0.9.0 with RxJava2.x support

2017-04-11 Android, DSP2017, Java, Open source, RxJava No comments

This time, I upgraded my another reactive Android open-source project called ReactiveNetwork to RxJava2.x. Many thanks goes to @tushar-acharya who performed initial migration to the newer version of RxJava. During migration, I’ve also created new package rx2 to avoid potential import conflicts during migration inside Android apps. Besides migration, I’ve updated sample apps, documentation & JavaDocs on Github pages. You can still use RxJava1.x version and it’s available on the branch with that name.

To use brand new ReactiveNetwork compatible with RxJava2.x, add the following dependency to your build.gradle file:

dependencies {
  compile 'com.github.pwittchen:reactivenetwork-rx2:0.9.0'
}

If you still want or need to use RxJava1.x, use the following dependency:

dependencies {
  compile 'com.github.pwittchen:reactivenetwork:0.9.0'
}

New updates and bug-fixes are on the way. Right now I have a few issues in the project backlog.

Feel free to contribute to this project and report new issues! Any constructive feedback will be appreciated.

ReactiveBeacons – release of v. 0.6.0 with support for RxJava2

2017-04-03 Android, Bluetooth Low Energy, DSP2017, Java, Open source, RxJava No comments

Thanks to @BugsBunnyBR I released new version of ReactiveBeacons library with the RxJava2.x support. It’s an Android library scanning BLE (Bluetooth Low Energy) beacons nearby with RxJava Observables. I also kept backward compatibility with RxJava1.x. Different versions of the libraries are located on the separate git branches. It’s a similar approach to original RxJava project. I have separate builds on Travis CI, separate artifacts and JavaDocs. Such approach generates more overhead, but in such case, RxJava1.x can be kept in a maintenance mode and RxJava2.x can be a subject of the future development.

What has been done in this version?

  • migrated library to RxJava2.x on RxJava2.x branch and released it as reactivebeacons-rx2 artifact
  • kept library compatible with RxJava1.x on a RxJava1.x branch and released it as reactivebeacons artifact
  • removed master branch
  • bumped library dependencies
  • added permission annotations
  • organized Gradle configuration
  • transformed instrumentation unit tests to pure java unit tests
  • started executing unit tests on Travis CI server
  • created separate JavaDoc for RxJava1.x and RxJava2.x

If you want to add RxJava2.x version to your Android project, add the following dependency to build.gradle file:

dependencies {
  compile 'com.github.pwittchen:reactivebeacons-rx2:0.6.0'
}

For RxJava1.x you can use old artifact id:

dependencies {
  compile 'com.github.pwittchen:reactivebeacons:0.6.0'
}

This library was one of the first experiments with my migrations to RxJava2.x. I have plans to migrate rest of my libraries soon.
Thanks to the awesome open-source community on GitHub, this process goes faster and I don’t have to do everything by myself.

SJUG updates – March 2017

2017-03-05 DSP2017, Java, Meetups No comments

SJUG aka Silesia Java Users Group meetups were reactivated one year ago and it’s still alive! I think it’s a great success of the community of Java developers who are located there. It’s not an easy thing to gather people from different cities in the region in one place every month, but it’s possible! During the year we gathered a few sponsors and partners, discounts for software tools and tickets for the conferences. Moreover, we had guests outside the Silesian region. There were really interesting presentations and discussions. Recently, I updated the website of the SJUG, so it automatically downloads information about the latest meetup. Now, we don’t have to do it manually every time.
Next meetup is planned for this Friday (10.03.2017) and if you’re interested in it, feel free to join.

More details about the group:

JDD 2016 Conference

2016-08-21 Conferences, Java No comments

Overview

This year in the beginning of the October is highly recommended to be in Kraków, Poland in order to attend JDD (Java Developers Days) Conference.

jdd_logoJDD is an unusual conference. You can hear the presentations of both experienced people and those who are starting their adventure in the IT industry. You meet speakers from Poland and all around the world. There’s also a wide range of presentation topics concerning not only technical trends and practices but also work methodology, communication, how to talk with people, etc. what has the same importance as coding proficiency.

JDD_4  JDD_1  JDD_7

Besides the presentations, you can meet a lot of interesting people from IT industry in Poland and abroad including programmers, QAs, managers, leaders, etc. There’s a great possibility to establish new contacts and meet old friends. If you are looking for new job opportunities or want to make a market research, you can also do that because there’s a lot of companies looking for new employees. I’ve attended this conference in the last years and I can tell that atmosphere is really great and you can learn a lot. Moreover, Kraków is a very nice city to spend some time, visit the old town, sightseeing, and go to the pubs in the night. Not to mention, there’s party after conference days :-).

Details

Discount for the blog readers!

All blog readers have 15% discount for the conference tickets with the code JDD_wittblog. I hope, it’ll encourage you to fill in the registration form.

See you at the conference!

Dockerizing Hybris

2016-07-27 Docker, Hybris, Java 1 comment

Introduction

A few months ago, I started work at Hybris, which was acquired by SAP, so our “division” is officially called SAP Hybris. I work in a team developing an extension for Hybris platform with assisting extensions and internal framework. There are many teams around the world developing their own extensions, which are finally integrated and packed into Commerce Suite provided to the clients and partners. We have our own development environment, but sometimes there’s need to build and run whole Commerce Suite in the case of system tests, bug reproduction, verification, etc. It’s not really hard, but you have to know what to do and you have to perform a lot of steps manually. This is kind of old approach in present days. Moreover, it’s really big project, so build and initialization process takes time. You have to download huge zipped package, unpack it, run Gradle script with installation recipe, compile project, run initialization and finally start the server.

Automating work

I wondered if it’s possible to automate mentioned process. Moreover, I wanted to learn more about Docker and create more advanced Dockerfile than “hello world”, which will be real life use case. That’s why I decided to dockerize SAP Hybris Commerce Suite. I called this project ydocker and you can find it at https://github.com/pwittchen/ydocker. Before you start doing anything, you need to remember to install Docker on your system. You can check out my notes about installation Docker on Linux in learning-docker repository.

Note #1: It’s not official company solution yet. Right now it’s just my personal proof of concept.

Note #2: I had problems with building and running this Docker container on OS X and I haven’t tested it on MS Windows. These systems need to use boot2docker, Docker Toolbox or another approach like that. I had no problems with it on Linux (Ubuntu 14.04 LTS), so this system is recommended if you want to build and run this container successfully. Probably other Linux distributions or Ubuntu versions will handle this as well.

Note #3 (update from 01.10.2016): Guys working on Docker improved their software for Mac, so it should work without any problems on this system now. It was tested on OS X El Capitan 10.11.6 and works fine. We just need to get Docker for Mac.

Building Docker container

Except for Dockerfile, ydocker repository also contains helper script ydocker, which has the following parameters:

-b    building Docker container
-r    running Hybris Server in Docker container
-c    running Docker container with CLI
-i    showing info about Docker container
-u    showing Commerce Suite Download Url
-d    deleting Docker container
-h    showing help

This script uses configuration file ydocker.conf, which has the following contents:

DOCKER_IMAGE_NAME=sap-hybris-commerce-suite
COMMERCE_SUITE_VERSION=latest
RECIPE=b2c_acc
HOST_PORT=9002
CONTAINER_PORT=9002

You can customize this configuration. E.g. choose a different version of the suite, different recipe or change server port.

To build container, you can just type:

./ydocker -b

Then, provide your credentials and Docker will:

  1. create container based on Ubuntu
  2. install Java 8
  3. set Java 8 as default Java version
  4. install wget
  5. download SAP Hybris Commerce Suite via wget
  6. unpack downloaded SAP Hybris Commerce Suite
  7. remove downloaded zipped package to save some disk space
  8. run configured installation recipe
  9. compile the project
  10. run system initialization

It may take some time. If you have good hardware setup it may take about 30 minutes.

Running Docker container

When everything is done, you can run docker image, with the command:

./ydocker -r

After that, Docker will start the server at localhost (127.0.0.1) and port 9002, which is exposed.
Commerce Suite will be started in a default configuration, which is not production ready, but is good for testing and demonstration purposes.
You need to wait for a few minutes to let the server warm up and then you can open administration console from the browser at http://127.0.0.1:9002.

In addition, if you want to browse generated configuration or unpacked suite, you can run Docker container with CLI with the following command:

./ydocker -c

It won’t start the server, but allow you to take a look around container via terminal.

Summary

That’s it! This proof of concept shows that we can get, build and run complicated and a huge project like SAP Hybris Commerce Suite inside Docker container, automate a lot of manual work and transform old school manual deployment and build process into elegant and standardized Docker container.