New blog domain: kula.blog

It's still work in progress but new posts will be published on  https://kula.blog/ Thanks to 11ty base blog RSS is working from the start so it should be easy to add to your reader if you still use any :) If now then I hope you can sign up to the newsletter where I'll be publishing new posts and interesting articles to it from time to time. Please use  kula.blog  from now on.

JVM Languages speed test: Java vs. Scala vs. Groovy

This is helloworld spped test... so it shows nothing about speed other than mentioned in comments. 
For all that doesn't like my test: For me it shows how small applications can work and if they are smart enough to ignore dummy code :) I've figured it after reading comments.


Summary:
Java is slow,
Scala works probably the same but starts longer.
Groovy is slower.
Groovy++ - maybe some day I will have time to test it. 

I read about Groovy and Scala, both languages are interesting. Groovy is easier and have much better support, it is like heaven and hell when you compare Scala and Groovy support (Intelli IDEA is only one that works for me... but I think that when I install in Eclipse Groovy plugin it breakes Scala support, Groovy and Scala are fighting in my computer and when one works second breaks, but it is only for me).
I like Groovy but i read about how slow it is, lets write veeery simple test and check it:
Scala:

object Sample{
  def main(args: Array[String]) = {
        for( g <- 1 to 10) { test()}
  }
  def test() = {
    val startTime = System.nanoTime();
    for(i <- 1 to 100){
        for(j <- 1 to 100){
          for(k <- 1 to 100){
            for(l <- 1 to 100){
                    i+j+k+l
            }
          }
        }
    }
    val time = System.nanoTime() - startTime
    System.out.println(time/1000000);
  }
}
Response:

4731
4729
4661
4871
4765
4618
4722
4805
5004
4768
Groovy


public class Sample{
  static main(args){
       for(i in 1..10){ test()}
  }
  static test(){
    long startTime = System.nanoTime();
    for(i in 1..100){
        for(j in 1..100){
          for(k in 1..100){
            for(l in 1..100){
                   i+j+k+l
            }
          }
        }
    }
    long time = System.nanoTime() - startTime
    System.out.println((int)time/1000000);
  }
}
Response:
16077
15462
16454
17798
17725
17814
16148
16081
16164
22489
Java:
public class Main {
    public static void main(String[] args){
         for(int g = 0; g<10; g++){ test(); }
    }
    static void test(){
           long startTime = System.nanoTime();
        for(int i = 1; i< 100; i++){
            for(int j = 1; j< 100; j++){
              for(int k = 1; k< 100; k++){
                for(int l = 1; l< 100; l++){
                      int m = i+j+k+l;
                }
              }
            }
        }
        long time = System.nanoTime() - startTime ;
        System.out.println(time/1000000);
    }
}
Response: 

191
175
170
184
182
173
176
168
168
169

































    1. All are from Intelli IDEA, as "Run Application".














  1. Bo be honest with Scala and Groovy I compiled them from command line and run compiled versions:
    Scala:

    4372
    4264
    4595
    4571
    4313
    4378
    4369
    4335
    4458
    4588
    Groovy:

    15854
    15728
    16917
    17265
    16141
    16504
    16155
    16849
    16007
    16496
















    1. Faster, but still so slow...



    Conclusion:








    1. 4731 - Scala




      16454 - Groovy


      176 - Java
      From compiled versions:
















    1. 15854 is 15,9 seconds...



















      1. 4588 is 4,6 seconds


















        1. 176 is so fast! ;) 0,176 s. 
          With some changes in Java code I get about 180-200.
          If I have errors in code please point them in comments.
          I must think about that numbers, maybe I have errors in code?
          I am waiting to responses with yours tests!
          Regards,
          Krzysztof Kula

Comments

  1. Language comparison tests are one of the hardest problems and writing a good test is not that simple. And there is no one ultimate, good tests to compare two languages.
    You always have to take into account what you actually want to test, operations and application type.

    In my opinion your test is not very accurate. The variable inside the internal loop is not used anywhere hence a good compiler may just remove the internal expression significantly increasing the loop speed.
    I don't know if any compilers for the above languages actually do that but if at least one do it (Java for example) then the above test compares compilers instead of the language.

    And this is not all. Please note, I do not know much about Scala but I know that Groovy doesn't use primitive types (long, int, boolean, arrays, etc...) instead it performs all operations on Objects (Long, Integer, Boolean, ArrayList, etc...). This causes the obvious slow down related to object allocation and obvious slower operations on objects than primitives.
    It also causes non-obvious slowdown related to much higher GC activity. The thing is that this GC activity can kick out much later, depending on how much memory you assigned to JVM it may even become apparent after a few minutes when the test is long over.
    Therefore the same test run over extended period of time may show different results from the test run over a few seconds only.

    Again, this is just a matter what really you are interested in. The client applications usually don't run for a very long time and GC activity normally doesn't matter. For server applications this is a different story.

    I think an interested modification to the test would be saving calculation results from the internal loop in some cyclic array of 1000 elements size. This way you could make sure the expression is actually executed and also you could add some array operations to the test. It seems to me that running each test for a couple of minutes could be interesting as at some point GC activity should start affecting the test and may slow down some runs.

    ReplyDelete
  2. Interesting what You write! Yes variable can be replaced by compilator, by in real time in code before/afrer many changes and bugfixes in a long time that unused code can live and slow down application. It is pleased to know that my compiler is smart enougth to remove it ;)

    About GC, yesterday I made myself a 'free day', and I don't know when I done another one. So If you done some comparision with Memory I will be interested to read it :)

    In title of my post is "Speed Test" and even with GC all values of code execution always must be averaged so in the end I think that results must me al least similar.

    About int and Integer in Groovy, as I know there is no option in Groovy to use int type. In this test I was trying to test speed with code similar to that I will write in real life, so I don't bother in optimising.

    If You do some test please get me know :)

    ReplyDelete
  3. Comparing Scala'a powerfull for comprehension to simple loop is just stupid. While loop will be mus faster.

    But, you have no fracking idea how to do benchmarks. HotSpot will remove all those stupid loops. So Java time will be just 0 (zero). LOL

    ReplyDelete
  4. http://www-128.ibm.com/developerworks/java/library/j-jtp02225.html?ca=drs-j0805

    So don't spread misinformation please...

    ReplyDelete
  5. Where are about run params? Ok. Truth is that You know nothing about how JVM works. First of all, when Running Your test with -server JVM option the results for Scala are:
    969
    745
    747
    736
    745
    745
    744
    725
    748
    743

    But for in Scala is something completely different than in Java.In Scala for is a Range that's why it's slower.

    here's upgraded version of Your "test" :

    def test() = {
    val startTime = System.nanoTime();
    var i, j, k, l = 0

    while(i < 10000) {
    i+=1
    while(j < 10000){
    j+=1
    while(k < 10000){
    k+=1
    while(l < 10000){
    l+=1
    i+j+k+l
    }
    }
    }
    }

    Now note that I multiply Your funny 100 by 100, and here're results:
    1
    1
    1
    1
    1
    1
    1
    1
    8
    0

    So please. Don't compare such garbage language like Groovy with Scala, and don't spread disinformation with such sloppy "benchmarks".

    Btw. I'm running MacBook pro with c2d 2.4Ghz, and I'm using Idea too.

    ReplyDelete
  6. Wow, we're just setting up a site ihavenoideahowtobenchmark.org, be sure you're gonna be there.

    ReplyDelete
  7. @dmilith About Scala results: Are You fanboy? Don;t get me wrong but that You proove only that 'while' is faster than 'for'.
    I never think of myself as an expert and that blog is not a place to share with the world with my thougths, and to learn from comments ;)

    @dmilith that garbage Groovy language with code like that yours:

    public class Sample{
    static main(args){
    for(i in 1..10){ test()}
    }
    static test(){

    long startTime = System.nanoTime();
    int i = 0, j=0, k=0, l=0, sum = 0;
    while(i < 100){
    i+=1;
    while(j < 100){
    j+=1;
    while(k < 100){
    k+=1;
    while(l < 100){
    l+=1;
    i+j+k+l;

    }
    }
    }
    }


    long time = System.nanoTime() - startTime
    System.out.println((int)time/1000000);
    }
    }
    Result:
    226
    3
    9
    8
    0
    0
    0
    0
    0
    0

    So only first loop in Groovy is slower. And nest results are successively faster.

    The reason of doing that simple test was to choose a language to learn, so I can tell that I have only basic knowledge about that languages! Language speed is important to me so I write those very simple tests.
    Java version of your code has:
    0
    0
    0
    0
    0
    0
    0
    0
    0
    0

    Can You tell me why in Scala for loop is a Range? I often use for loops and want them to be fast and efficient.
    But ofter these test I probably start using while loops ;)

    ReplyDelete
  8. While I said "garbage language" I did want to say that Groovy has mess in code and ideology of whole language.
    Ofcourse Groovy will have same result.. Like Anonimowy said before, hotspot will remove all those loops.. No matter which JVM based language You'll use.. It will return similar results every time.
    Note that for loops has just different purpose in Scala. They're more powerful than in Java for example.

    And no I'm not fanboy. I know some weak sides of Scala (f.e. boxing of primitives, incompatibility with List and Iterators from Java which is caused different purpose of Lists in Scala, etc.) but in comparison with rest of JVM languages it's the best choice for productive and effective programming

    (It's my opinion ofcourse)

    ReplyDelete
  9. Your code is not counting like mine, try to fix it and then post results :)

    Hint: what about values of j,k,l after first loop? ;)

    I still cant decide... Maybe I will learn only additional Java library instead of new Language. Even so I think about : Groovy, Scala, Python, Ruby, C#(I know that it is gaining popularity and friend told me that ti is used in AGH for students projects).

    ReplyDelete
  10. Na AGH zwykle nie ma przymusu użycia konkretnego języka, więc nie wiem co to za argument. To samo mogę powiedzieć o Scali, Pythonie czy Ruby.
    Łatwiej rzucić kostką niż wybierać język na podstawie zagnieżdżonych pętli (zwłaszcza że tylko z pozoru porównujesz to samo), rezultat będzie podobny.

    ReplyDelete
  11. Chodziło mi raczej o przeprowadzenie testu na własną rękę którym by mi dawał jakieś orientacyjne pojęcie jak z tą szybkością jest. Czytałem ostatnio o tym jaki Groovy jest wolny postanowiłem to sprawdzić na własną rękę, nie nadaję temu nie-wiadomo-jakiej wagi i ostateczności :)

    Co do C# to muszę dopytać dla pewności ale kolega jest na uzupełniających magisterskich i ma projekty np. w Javie, ASP(C#) jako że on nie zna żadnego z tych języków na 95% mogę powiedzieć że takiej dowolności nie miał, no może mógł pisać w Javie albo w C# ;)

    ReplyDelete
  12. Scala ma wydajność prawie taką samą jak Java (czasem trochę wolniej, czasem trochę szybciej), a ty się naucz wpierw pisać mikrobenchmarki bo porównywanie pustych pętli jest bez sensu (HotSpot je wytnie i ten kod się po prostu nie wykona). Także porównywanie form comprehension z prymitywną pętlą Javy jest kompletnie bez sensu. Scalowe "for" to dużo więcej niż pętla.

    Żaden tam Groovy, Python, Ruby czy PHP nie podkoczy wydajnościowo do Scali, to rzecz oczywista. Co do Grooviego to jego twórca sam powiedział, że gdyby wcześniej poznał Scalę to Groovy by nigdy nie powstał. :-D Zresztą po co pisać w Groovy? Dla Grails? Toż to pokraczna kopia Rails. Co innego Lift (napisany w Scali), ale to wyższa poprzeczka.

    W tym wszystkim nie wspomniano o bardzo potężnym i ciekawym języku jakim jest Clojure. Jego potęga to potęga Lispa gdzie dane są kodem a kod danymi (Clojure jest homoiconic). Poza tym Clojure jest też najwydajniejszym, dynamicznie typowanym językiem dla JVM. Ogólnie polecam doskonała prezentację http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey oraz 2 czesc: http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey. Takie prezentacje trochę otwierają oczy. Paul Graham miał rację pisząć, że tylko ci co poznali najpotężniejszy z języków programowania są w stanie porównywać inne języki. Reszta nie wie po prostu o czym mówi. Obowiązkowa lektura: "Beating the Averages" http://www.paulgraham.com/avg.html

    ReplyDelete
  13. 'while' is faster than 'for' - We need to reinvent the whole of computer science again.

    ReplyDelete
  14. Niedawno kończyłem AGH i projekty pisałem praktycznie w dowolnym języku, takim który był wg mnie najodpowiedniejszy (czyt. najszybciej było skończyć projekt) do mojego zadania. Problemy były jedynie wtedy kiedy prowadzący nie chciał zestawić środowiska dla mojego projektu, więc albo musiałem o to zadbać sam (głównie wykorzystując laptop) albo zrezygnować i pisać w języku pod który infrastruktura była już przygotowana. Tak naprawde mało kogo obchodził kod, dlatego ta dowolność technologii/języka.

    Co do benczmarku to wszystko byłoby OK gdybyś testował te same rzeczy, bo nawet microbenchmark daje jakieś wyniki do interpretacji. Ale w obecnej sytuacji wprowadzasz w błąd siebie i innych.

    ReplyDelete
  15. Widzę, że kolega Krzychu nie tylko nie ma pojęcia o Scali ani o benchmarkowaniu Javy ale także nie za bardzo umie pisać w Javie. HotSpot dla Javy, podobnie jak dla Scali, musi tu zwrócić zera, bo te pętle zostaną po prostu wyrzucone z bytecodu.

    http://gist.github.com/282504#file_dupa_nie_test.java

    ReplyDelete
  16. I have a nasty feeling that you have no idea how to write a benchmark test and that your knowledge of Scala is really poor. This post ist just a rubbish.

    ReplyDelete

Post a Comment

Comments:

Popular posts from this blog

How to use NPM packages from private repositories on bitbucket

How to simulate slow connection (developer proxy in Node.js)