Planet Classpath

The IcedTea project provides a harness to build the source code from OpenJDK using Free Software build tools, along with additional features such as the ability to build against system libraries and support for alternative virtual machines and architectures beyond those supported by OpenJDK.

This release updates our OpenJDK 7 support in the 2.6.x series with the October 2016 security fixes from OpenJDK 7 u121.

If you find an issue with the release, please report it to our bug database under the appropriate component. Development discussion takes place on the distro-pkg-dev OpenJDK mailing list and patches are always welcome.

Full details of the release can be found below.

What’s New?

New in release 2.6.8 (2016-11-13)

  • Security fixes
  • Import of OpenJDK 7 u121 build 0
    • S6624200: Regression test fails: test/closed/javax/swing/JMenuItem/4654927/bug4654927.java
    • S6882559: new JEditorPane(“text/plain”,”") fails for null context class loader
    • S7090158: Networking Libraries don’t build with javac -Werror
    • S7125055: ContentHandler.getContent API changed in error
    • S7145960: sun/security/mscapi/ShortRSAKey1024.sh failing on windows
    • S7187051: ShortRSAKeynnn.sh tests should do cleanup before start test
    • S8000626: Implement dead key detection for KeyEvent on Linux
    • S8003890: corelibs test scripts should pass TESTVMOPTS
    • S8005629: javac warnings compiling java.awt.EventDispatchThread and sun.awt.X11.XIconWindow
    • S8010297: Missing isLoggable() checks in logging code
    • S8010782: clean up source files containing carriage return characters
    • S8014431: cleanup warnings indicated by the -Wunused-value compiler option on linux
    • S8015265: revise the fix for 8007037
    • S8016747: Replace deprecated PlatformLogger isLoggable(int) with isLoggable(Level)
    • S8020708: NLS mnemonics missing in SwingSet2/JInternalFrame demo
    • S8024756: method grouping tabs are not selectable
    • S8026741: jdk8 l10n resource file translation update 5
    • S8048147: Privilege tests with JAAS Subject.doAs
    • S8048357: PKCS basic tests
    • S8049171: Additional tests for jarsigner’s warnings
    • S8059177: jdk8u40 l10n resource file translation update 1
    • S8075584: test for 8067364 depends on hardwired text advance
    • S8076486: [TESTBUG] javax/security/auth/Subject/doAs/NestedActions.java fails if extra VM options are given
    • S8077953: [TEST_BUG] com/sun/management/OperatingSystemMXBean/TestTotalSwap.java Compilation failed after JDK-8077387
    • S8080628: No mnemonics on Open and Save buttons in JFileChooser
    • S8083601: jdk8u60 l10n resource file translation update 2
    • S8140530: Creating a VolatileImage with size 0,0 results in no longer working g2d.drawString
    • S8142926: OutputAnalyzer’s shouldXXX() calls return this
    • S8143134: L10n resource file translation update
    • S8147077: IllegalArgumentException thrown by api/java_awt/Component/FlipBufferStrategy/indexTGF_General
    • S8148127: IllegalArgumentException thrown by JCK test api/java_awt/Component/FlipBufferStrategy/indexTGF_General in opengl pipeline
    • S8150611: Security problem on sun.misc.resources.Messages*
    • S8157653: [Parfait] Uninitialised variable in awt_Font.cpp
    • S8158734: JEditorPane.createEditorKitForContentType throws NPE after 6882559
    • S8159684: (tz) Support tzdata2016f
    • S8160934: isnan() is not available on older MSVC compilers
    • S8162411: Service Menu services 2
    • S8162419: closed/com/oracle/jfr/runtime/TestVMInfoEvent.sh failing after JDK-8155968
    • S8162511: 8u111 L10n resource file updates
    • S8162792: Remove constraint DSA keySize < 1024 from jdk.jar.disabledAlgorithms in jdk8
    • S8164452: 8u111 L10n resource file update – msgdrop 20
    • S8165816: jarsigner -verify shows jar unsigned if it was signed with a weak algorithm
    • S8166381: Back out changes to the java.security file to not disable MD5
  • Backports
    • S6604109, PR3162: javax.print.PrintServiceLookup.lookupPrintServices fails SOMETIMES for Cups
    • S6907252, PR3162: ZipFileInputStream Not Thread-Safe
    • S8024046, PR3162: Test sun/security/krb5/runNameEquals.sh failed on 7u45 Embedded linux-ppc*
    • S8028479, PR3162: runNameEquals still cannot precisely detect if a usable native krb5 is available
    • S8034057, PR3162: Files.getFileStore and Files.isWritable do not work with SUBST’ed drives (win)
    • S8038491, PR3162: Improve synchronization in ZipFile.read()
    • S8038502, PR3162: Deflater.needsInput() should use synchronization
    • S8059411, PR3162: RowSetWarning does not correctly chain warnings
    • S8062198, PR3162: Add RowSetMetaDataImpl Tests and add column range validation to isdefinitlyWritable
    • S8066188, PR3162: BaseRowSet returns the wrong default value for escape processing
    • S8072466, PR3162: Deadlock when initializing MulticastSocket and DatagramSocket
    • S8075118, PR3162: JVM stuck in infinite loop during verification
    • S8076579, PR3162: Popping a stack frame after exception breakpoint sets last method param to exception
    • S8078495, PR3162: End time checking for native TGT is wrong
    • S8078668, PR3162: jar usage string mentions unsupported option ‘-n’
    • S8080115, PR3162: (fs) Crash in libgio when calling Files.probeContentType(path) from parallel threads
    • S8081794, PR3162: ParsePosition getErrorIndex returns 0 for TimeZone parsing problem
    • S8129957, PR3162: Deadlock in JNDI LDAP implementation when closing the LDAP context
    • S8130136, PR3162: Swing window sometimes fails to repaint partially when it becomes exposed
    • S8130274, PR3162: java/nio/file/FileStore/Basic.java fails when two successive stores in an iteration are determined to be equal
    • S8132551, PR3162: Initialize local variables before returning them in p11_convert.c
    • S8133207, PR3162: [TEST_BUG] ParallelProbes.java test fails after changes for JDK-8080115
    • S8133666, PR3162: OperatingSystemMXBean reports abnormally high machine CPU consumption on Linux
    • S8135002, PR3162: Fix or remove broken links in objectMonitor.cpp comments
    • S8137121, PR3162: (fc) Infinite loop FileChannel.truncate
    • S8137230, PR3162: TEST_BUG: java/nio/channels/FileChannel/LoopingTruncate.java timed out
    • S8139373, PR3162: [TEST_BUG] java/net/MulticastSocket/MultiDead.java failed with timeout
    • S8140249, PR3162: JVM Crashing During startUp If Flight Recording is enabled
    • S8141491, PR3160, G592292: Unaligned memory access in Bits.c
    • S8144483, PR3162: One long Safepoint pause directly after each GC log rotation
    • S8149611, PR3160, G592292: Add tests for Unsafe.copySwapMemory
  • Bug fixes
    • S8078628, PR3151: Zero build fails with pre-compiled headers disabled
    • PR3128: pax-mark-vm script calls “exit -1″ which is invalid in dash
    • PR3131: PaX marking fails on filesystems which don’t support extended attributes
    • PR3135: Makefile.am rule stamps/add/tzdata-support-debug.stamp has a typo in add-tzdata dependency
    • PR3141: Pass $(CC) and $(CXX) to OpenJDK build
    • PR3166: invalid zip timestamp handling leads to error building bootstrap-javac
    • PR3202: Update infinality configure test
    • PR3212: Disable ARM32 JIT by default
  • CACAO
    • PR3136: CACAO is broken due to 2 new native methods in sun.misc.Unsafe (from S8158260)
  • JamVM
    • PR3134: JamVM is broken due to 2 new native methods in sun.misc.Unsafe (from S8158260)
  • AArch64 port
    • S8167200, PR3204: AArch64: Broken stack pointer adjustment in interpreter
    • S8168888: Port S8160591: Improve internal array handling to AArch64.
    • PR3211: AArch64 build fails with pre-compiled headers disabled

The tarballs can be downloaded from:

We provide both gzip and xz tarballs, so that those who are able to make use of the smaller tarball produced by xz may do so.

The tarballs are accompanied by digital signatures available at:

These are produced using my public key. See details below.

  • PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
  • Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222

GnuPG >= 2.1 is required to be able to handle this key.

SHA256 checksums:

  • e5f0a14077de47a1e6bcba672042880f1cb28859468fe95570555593a28fe02b icedtea-2.6.8.tar.gz
  • 65628538255b2657b1228b534c18ffb74e52be11d1d25cf694d02f39efabf70d icedtea-2.6.8.tar.gz.sig
  • 854030ff1b580d896dbabbdb0e64dc0ef3537786285808a7b3cdfcb80520255d icedtea-2.6.8.tar.xz
  • 23336d9d5aa7256cfc267f9b86eb46b69e0439af3b479405d215f12932ebbe63 icedtea-2.6.8.tar.xz.sig

The checksums can be downloaded from:

A 2.6.8 ebuild for Gentoo is available.

The following people helped with these releases:

We would also like to thank the bug reporters and testers!

To get started:

$ tar xzf icedtea-2.6.8.tar.gz

or:

$ tar x -I xz -f icedtea-2.6.8.tar.xz

then:

$ mkdir icedtea-build
$ cd icedtea-build
$ ../icedtea-2.6.8/configure
$ make

Full build requirements and instructions are available in the INSTALL file.

Happy hacking!

A year or so ago I was asked to debug a crash in the Firefox devtools.  Crashes are easy!  I fired up gdb and reproduced the crash… which turned out to be in some code JITted by SpiderMonkey.  I was immediately lost; even a simple bt did not work.  Someone more familiar with the JIT — hi Shu — had to dig out the answer :-(.

I did take the opportunity to get some information from him about how he found the result, though.  He pointed me to the code responsible for laying out JIT stack frames.  It turned out that gdb could not unwind through JIT frames, but it could be done by hand — so I resolved then to eventually fix this.

Phase One

I knew from my gdb hacking that gdb has a JIT unwinding API.  Actually — and isn’t this the way most programs end up working? — it has two.

The first JIT API requires some extra work on the part of the JIT: it constructs an object file, typically ELF and DWARF, in memory, then calls a hook.  GDB sets a breakpoint on this hook and, when hit, it reads the data from the inferior.  This lets the JIT provide basically any kind of information — but it’s pretty heavy.

So, I focused my attention on the second API.  In this mode, the JIT author would provide a shared library that used some callbacks to inform gdb of the details of what was going on.  The set of callbacks was much more limited, but could at least describe how to unwind the registers.  So, I figured that this is what I would do.

But… I didn’t really want to write this in C.  That would be a real pain!  C is fiddly and hard to deal with, and it would mean constant rebuilding of the shared library while debugging, and SpiderMonkey already had a reasonable number of gdb-python scripts — surely this could be done in Python.

So I took the quixotic approach, namely writing a shared library that used the second gdb JIT API but only to expose this API to Python.

Of course, this turned out to be Rube Goldbergian.  Various parts of the gdb Python API could not be called from the JIT shared library, because those bits depended on other state in gdb, which wasn’t set properly when the JIT library was being called.  So, I had gdb calling into my shared library, which called my Python code, which then invoked a new gdb command (written in Python and supplied by my package) — that existed solely for the purpose of setting this internal state properly — and that in turn invoked the code I wanted to run, say to fetch memory or a register or something.

Computer Science!

Well, that took a while.  But it sort of worked!  And maybe I could just keep it in github and not put it in Mozilla Central and avoid learning about the Firefox build system and copying in some gdb header file and license review and whatnot.

So I started writing the actual Python code… OMG.  And see below since you will totally want to know about this.  But meanwhile…

… while I was hacking away on this crazy idea, someone implemented the much more sane idea of just exposing gdb’s unwinder API to gdb’s Python layer.

Hmm… why didn’t I do that?  Well, I left gdb under a bit of a cloud, and didn’t really want to be that involved at the time.  Plus, you know, gdb is a high quality project; which means that if you write a giant patch to expose the unwinding API, you have to be prepared for 17 rounds of patch review (this really happened once), plus writing documentation and tests.  Sometimes it’s just easier to channel one’s inner Rube.

Phase Two

The integrated Python API was a great development.  Now I could delete my shared library and my insane trampoline hacks, and focus on my insane unwinding code.

A lot of this work was straightforward, in the sense that the general outline was clear and just the details remained.  The details amount to things like understanding the SpiderMonkey frame descriptor (which partly describes the previous frame and partly the new frame; there’s one comment explaining this that somehow eluded me for quite a while); duplicating the SpiderMonkey JIT unwinding code in Python; and of course carefully reading the SpiderMonkey code that JITs the “entry frame” code to understand how registers are spilled.

Naturally, while doing this it turned out that I was maybe the first person to use these gdb APIs in anger.  I found some gdb crashes, oops!  The docs would have been impenetrable, except I already knew the underlying C APIs on which they were based… whew!  The Python API was unexpectedly picky in other areas, too.

But then there was also some funny business, one part in gdb, and one part in SpiderMonkey.

GDB is probably more complicated than you realize.  In this case, the complexity is that, in gdb, each stack frame can have its own architecture.  This seemingly weird functionality is actually used; I think it was invented for the SPU, but some other chips have multiple modes as well.  But what this means is that the question “what architecture is this program?” is not well-defined, and anyway gdb’s Python layer doesn’t provide you a way to find whatever approximation it is that would make sense in your specific case.  However, when writing the SpiderMonkey unwinder, it kind of actually is well-defined and we’d like to know the answer to know which unwinder to choose.

For this problem I settled on the probably terrible idea of checking whether a given register is available.  That is, if you see “$rip“, you can guess it’s x86-64.

The other problem here is that gdb thinks that, since you wrote an unwinder, it should get the first stab at unwinding.  That’s very polite!  But for SpiderMonkey, deciding “hey, is this PC in some code the JIT emitted?” is actually a real pain, or at least outside the random bits of it I learned in order to make all this work.

Aha!  I know, there’s probably a Python API to say “is this address associated with some shared library?”  I remembered reading and/or reviewing a patch… but no, gdb.solib_name is close but doesn’t do the right thing for addresses in the main executable.  WAT.

I tried several tricks without success, and in the end I went with parsing /proc/maps to get the mappings to decide whether a given frame should be handled by this unwinder or by gdb.  Horrible.  And fails with remote debugging.

Luckily, nobody does remote debugging.

Remote Debugging

Oh, wait, people do remote debugging at Mozilla all the time.  They don’t call it “remote debugging” though — they call it “using RR“, which while it runs locally, appears to be remote to gdb; and, importantly, during replay mode fakes the PID, and does other deep magic, though not deep enough to extend to making a fake map file that could be read via gdb’s remote get command.

By the way, you should be using RR.  It’s the best advance in debugging since, well, gdb.  It’s a process record-and-replay program, but unlike gdb’s built-in reverse debugging, it handles threads properly and has decent performance.

Oh Well

Oh well.  It just won’t work remotely.  Or at least not until fellow Mozillian (this always seems like it should be “Mozillan” to me, but it’s not, there really is that extra “i”) and all-star Nicolas Pierron wrote some additional Python to read some SpiderMonkey tables to make the decision in a more principled way.  Now it will all work!

Though looking now I wonder if I dreamed this, because the code isn’t checked in.  I know he had a patch but my memory is a bit fuzzy — maybe in the end it didn’t work, because RR didn’t implement the qGetTLSAddr packet, which gdb uses to read thread-local storage.  Did I mention the thread-locals?

The Real Start of the Story

So, way back at the beginning, during my initial foray into this code, I found that a crucial bit of information — the appropriately-named TlsPerThreadData — was stashed away in a thread-local variable.  Information stored here is needed by the unwinder in order to unwind from a C++ frame into a JIT frame.

Only, Firefox didn’t use “real” thread-local variables, the things that so many glibc and gcc hackers put so much effort into micro-optimizing.  No, it just used a template class that wrapped pthread_setspecific and friends in a relatively ergonomic way.

Naturally, for an unwinder this is a disaster.  Why?  Unwinding is basically the dissection of the stack; but in order to compute the value of one of these thread-local-storage objects, the unwinder would have to make some function calls in the inferior (in fact this prevents it from working on OSX).  But these would affect the stack, and also potentially let other inferior code (in other threads — remember, gdb is complicated and you can exert various unusual kinds of control like this) run as well.

So I neglected to mention the very first step: changing Firefox to use __thread.  (Ok, I didn’t really neglect to mention it, I was just being lazy and anyway it’s a shaggy dog story.)

Do Not Use libthread_db

RR did not implement qGetTLSAddr, which we needed, because  lots of people at Mozilla use RR.  So I set out to implement that.  This meant a foray into the dangerous world of libthread_db.

For reasons I do not know, and suspect that I do not want to know, glibc has historically followed many Solaris conventions.  One such Solaris innovation was libthread_db — a library that debuggers use to find certain information from libc, information like the address of a thread-local variable

On the surface this seems like a great idea: don’t bake the implementation details of the C library into the debugger.  Instead, let the debugger use a debugging library that comes with the C library.  And, if you designed it that way, it would be a good idea.

Sadly, though, libthread_db was not designed that way.  Oh no.

For example, libthread_db has a callback interface.  The calling program — gdb or rr — must provide some functions that libthread_db can call, to do some simple things like “read some memory”; or some very complicated things like “find the address of a symbol given its name”.  Normal C programmers might implement these callbacks using a structure containing function pointers.  But not libthread_db!  Instead it uses fixed symbol names that must be provided by the calling application.  Not all of these are required for it to work (you get to figure out which, yay!), but some definitely are.  And, you have to dlopen a libthread_db that matches the libc of the inferior that you’re debugging (or link against it, but that’s also obviously bad).

Wait, you say.  Doesn’t that mess up cross-debugging?  Why yes!  Yes it does!  Which is why qGetTLSAddr has to be in the gdb remote serial protocol to start with.

Hey, maybe the Linux vendors should fix this.  They are — see Gary Benson’s Infinity project — but unfortunately that’s still in development and I wanted RR to work sooner.

Ok, so whew.  I wrote qGetTLSAddr support for RR.  This was a small patch in the end, but an unusual pain in an already painful series.  Hopefully this won’t spill out into other programs.

glibc

Hahaha, you are so funny.  Of course it spills out: remember how you have to define a bunch of functions with specific names in your program in order to use libthread_db?  Well, how do you know you got the types correct?

Yeah, you include <proc_service.h> (a name deliberately chosen to confuse, I suppose, why not, it doesn’t bear any obvious relationship to the library).  Only, that was never installed by glibc.  Instead, gdb just copied it into the source tree.

So naturally I went and fixed this in glibc.  And, even more naturally, this broke the gdb build, which was autoconf’d to check for a file that never existed in the past.  LOL.

Thank You Cthulhu

At this point I figured it was only a matter of time until I had to patch the kernel.  Thankfully this hasn’t been necessary yet.

It Says What

In gdb the actual unwinding and the display of frames are separate concerns.

And let me digress here to say that gdb’s unwinder design is excellent.  I believe it was redone by Andrew Cagney (this was well before my active time in gdb, so apologies if you’re reading this and you did it and I’ve misattributed it).  Like much of gdb, many of the details are bizarre and take one back to the byte-counting days of 1987; but the high level design is very solid and has endured with, I think, just one significant change (to support inline functions) in the intervening 15 or so years.  I’ve long thought that this is a remarkable accomplishment in the programming world.

So, yes.  It’s not enough to just unwind.  Simply having an unwinder yields backtraces with lines like:

#5 0xfeefee ???

Better than nothing!  But not yet great.

The second part of the SpiderMonkey unwinder is, therefore, a gdb “frame filter”.  This is an object that takes raw frames and decorates them with information like a function name, or a file name, or arguments.

Work to add this information is ongoing — I landed one patch just yesterday, and another one, to add more information about interpreted frames, is still in the works.  And there are two more bugs filed… maybe this project, like this blog post, will never conclude.  It will just scroll endlessly.

But now, with all the code in place, bt can show something like:

#6 0x00007ffff7ff20f3 in <<JitFrame_BaselineJS "f1">> (this=JSVAL_VOID, arg1=$jsval(4700))

This is the call f1(4700).

Let’s Just Have One More

Of course we still couldn’t enable this unwinder by default.  You have to enable it by hand.

And by the way, in the first release of gdb’s Python unwinder feature, enabling or disabling an unwinder didn’t flush the frame cache, so it wouldn’t actually take effect until some invisible-to-the-user state change took place.  I fixed this bug, but here Pedro Alves also taught me the secret gdb command flushregs, which in fact just flushes the frame cache. (I’m going to go out on a limb and guess that this command predates the already ancient maint prefix command, hence its weird name.)

Anyway, you have to enable it by hand because the unwinder itself doesn’t work properly if the outermost frame is in JIT code.  The JIT, in the interest of performance, doesn’t maintain a frame pointer.  This means that in the outermost frame, there’s no reliable way to find the object that describes this frame and links to the previous frame.

Now, normally in this case gdb would either resort to debug info (not available here), or in extremis its encyclopedic suite of prologue analyzers (yes, gdb can analyze common function prologues for all architectures developed in the last 25 years to figure out stuff) — but naturally JIT compilers go their own way here as well.

Humans, like Shu back at the start of this story, can do this by dumping parts of the stack and guessing which bytes represent the frame header.

But, I’ve been reluctant and a bit afraid to hack a heuristic into the unwinder.

To sum up — in case you missed it — this means that all the code written during this entire saga would still not have helped with my original bug.

The End

We are pleased to announce the release of IcedTea 3.2.0!

The IcedTea project provides a harness to build the source code from OpenJDK using Free Software build tools, along with additional features such as the ability to build against system libraries and support for alternative virtual machines and architectures beyond those supported by OpenJDK.

This release updates our OpenJDK 8 support with the October 2016 security fixes from OpenJDK 8 u111. It also adds a number of features familiar from IcedTea 2.x:

  • Support for toggling the inclusion of native (--disable-native-debuginfo) and Java (--disable-java-debuginfo) debugging information.
  • Support for splitting native debuginfo into separate files (--enable-split-debuginfo)
  • Allow linking against the system Kerberos installation in order to obtain the cache location. (--enable-system-kerberos)
  • Allow linking against the system libpcsclite at compile-time. (--enable-system-pcsc)
  • Allow linking against the system libsctp at compile-time. (--enable-system-sctp)

and introduces a number of new features:

  • Support for building without pre-compiled headers (--disable-precompiled-headers)
  • The ability to use the system cryptography policies provided by the crypto-policies package on Fedora.

If you find an issue with the release, please report it to our bug database under the appropriate component. Development discussion takes place on the distro-pkg-dev OpenJDK mailing list and patches are always welcome.

Full details of the release can be found below.

What’s New?

New in release 3.2.0 (2016-11-08)

  • Security fixes
  • New features
    • PR1370: Provide option to build without debugging
    • PR1375: Provide option to strip and link debugging info after build
    • PR1537: Handle alternative Kerberos credential cache locations
    • PR1978: Allow use of system PCSC
    • PR2445: Support system libsctp
    • PR3182: Support building without pre-compiled headers
    • PR3183: Support Fedora/RHEL system crypto policy
    • PR3221: Use pkgconfig to detect Kerberos CFLAGS and libraries
  • Import of OpenJDK 8 u102 build 14
    • S4515292: ReferenceType.isStatic() returns true for arrays
    • S4858370: JDWP: Memory Leak: GlobalRefs never deleted when processing invokeMethod command
    • S6976636: JVM/TI test ex03t001 fails assertion
    • S7185591: jcmd-big-script.sh ERROR: could not find app’s Java pid.
    • S8017462: G1: guarantee fails with UseDynamicNumberOfGCThreads
    • S8034168: ThreadMXBean/Locks.java failed, blocked on wrong object
    • S8036006: [TESTBUG] sun/tools/native2ascii/NativeErrors.java fails: Process exit code was 0, but error was expected.
    • S8041781: Need new regression tests for PBE keys
    • S8041787: Need new regressions tests for buffer handling for PBE algorithms
    • S8043836: Need new tests for AES cipher
    • S8044199: Tests for RSA keys and key specifications
    • S8044772: TempDirTest.java still times out with -Xcomp
    • S8046339: sun.rmi.transport.DGCAckHandler leaks memory
    • S8047031: Add SocketPermission tests for legacy socket types
    • S8048052: Permission tests for setFactory
    • S8048138: Tests for JAAS callbacks
    • S8048147: Privilege tests with JAAS Subject.doAs
    • S8048356: SecureRandom default provider tests
    • S8048357: PKCS basic tests
    • S8048360: Test signed jar files
    • S8048362: Tests for doPrivileged with accomplice
    • S8048596: Tests for AEAD ciphers
    • S8048599: Tests for key wrap and unwrap operations
    • S8048603: Additional tests for MAC algorithms
    • S8048604: Tests for strong crypto ciphers
    • S8048607: Test key generation of DES and DESEDE
    • S8048610: Implement regression test for bug fix of 4686632 in JCE
    • S8048617: Tests for PKCS12 read operations
    • S8048618: Tests for PKCS12 write operations.
    • S8048619: Implement tests for converting PKCS12 keystores
    • S8048624: Tests for SealedObject
    • S8048819: Implement reliability test for DH algorithm
    • S8048820: Implement tests for SecretKeyFactory
    • S8048830: Implement tests for new functionality provided in JEP 166
    • S8049237: Need new tests for X509V3 certificates
    • S8049321: Support SHA256WithDSA in JSSE
    • S8049429: Tests for java client server communications with various TLS/SSL combinations.
    • S8049432: New tests for TLS property jdk.tls.client.protocols
    • S8049814: Additional SASL client-server tests
    • S8050281: New permission tests for JEP 140
    • S8050370: Need new regressions tests for messageDigest with DigestIOStream
    • S8050371: More MessageDigest tests
    • S8050374: More Signature tests
    • S8050427: LoginContext tests to cover JDK-4703361
    • S8050460: JAAS login/logout tests with LoginContext
    • S8050461: Tests for syntax checking of JAAS configuration file
    • S8054278: Refactor jps utility tests
    • S8055530: assert(_exits.control()->is_top() || !_gvn.type(ret_phi)->empty()) failed: return value must be well defined
    • S8055844: [TESTBUG] test/runtime/NMT/VirtualAllocCommitUncommitRecommit.java fails on Solaris Sparc due to incorrect page size being used
    • S8059677: Thread.getName() instantiates Strings
    • S8061464: A typo in CipherTestUtils test
    • S8062536: [TESTBUG] Conflicting GC combinations in jdk tests
    • S8065076: java/net/SocketPermission/SocketPermissionTest.java fails intermittently
    • S8065078: NetworkInterface.getNetworkInterfaces() triggers intermittent test failures
    • S8066871: java.lang.VerifyError: Bad local variable type – local final String
    • S8068427: Hashtable deserialization reconstitutes table with wrong capacity
    • S8069038: javax/net/ssl/TLS/TLSClientPropertyTest.java needs to be updated for JDK-8061210
    • S8069253: javax/net/ssl/TLS/TestJSSE.java failed on Mac
    • S8071125: Improve exception messages in URLPermission
    • S8072081: Supplementary characters are rejected in comments
    • S8072463: Remove requirement that AKID and SKID have to match when building certificate chain
    • S8072725: Provide more granular levels for GC verification
    • S8073400: Some Monospaced logical fonts have a different width
    • S8073872: Schemagen fails with StackOverflowError if element references containing class
    • S8074931: Additional tests for CertPath API
    • S8075286: Additional tests for signature algorithm OIDs and transformation string
    • S8076486: [TESTBUG] javax/security/auth/Subject/doAs/NestedActions.java fails if extra VM options are given
    • S8076545: Text size is twice bigger under Windows L&F on Win 8.1 with HiDPI display
    • S8076995: gc/ergonomics/TestDynamicNumberOfGCThreads.java failed with java.lang.RuntimeException: ‘new_active_workers’ missing from stdout/stderr
    • S8079138: Additional negative tests for XML signature processing
    • S8081512: Remove sun.invoke.anon classes, or move / co-locate them with tests
    • S8081771: ProcessTool.createJavaProcessBuilder() needs new addTestVmAndJavaOptions argument
    • S8129419: heapDumper.cpp: assert(length_in_bytes > 0) failed: nothing to copy
    • S8130150: Implement BigInteger.montgomeryMultiply intrinsic
    • S8130242: DataFlavorComparator transitivity exception
    • S8130304: Inference: NodeNotFoundException thrown with deep generic method call chain
    • S8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata
    • S8133023: ParallelGCThreads is not calculated correctly
    • S8134111: Unmarshaller unmarshalls XML element which doesn’t have the expected namespace
    • S8135259: InetAddress.getAllByName only reports “unknown error” instead of actual cause
    • S8136506: Include sun.arch.data.model as a property that can be queried by jtreg
    • S8137068: Tests added in JDK-8048604 fail to compile
    • S8139040: Fix initializations before ShouldNotReachHere() etc. and enable -Wuninitialized on linux.
    • S8139581: AWT components are not drawn after removal and addition to a container
    • S8141243: Unexpected timezone returned after parsing a date
    • S8141420: Compiler runtime entries don’t hold Klass* from being GCed
    • S8141445: Use of Solaris/SPARC M7 libadimalloc.so can generate unknown signal in hs_err file
    • S8141551: C2 can not handle returns with inccompatible interface arrays
    • S8143377: Test PKCS8Test.java fails
    • S8143647: Javac compiles method reference that allows results in an IllegalAccessError
    • S8144144: ORB destroy() leaks filedescriptors after unsuccessful connection
    • S8144593: Suppress not recognized property/feature warning messages from SAXParser
    • S8144957: Remove PICL warning message
    • S8145039: JAXB marshaller fails with ClassCastException on classes generated by xjc
    • S8145228: Java Access Bridge, getAccessibleStatesStringFromContext doesn’t wrap the call to getAccessibleRole
    • S8145388: URLConnection.guessContentTypeFromStream returns image/jpg for some JPEG images
    • S8145974: XMLStreamWriter produces invalid XML for surrogate pairs on OutputStreamWriter
    • S8146035: Windows – With LCD antialiasing, some glyphs are not rendered correctly
    • S8146192: Add test for JDK-8049321
    • S8146274: Thread spinning on WeakHashMap.getEntry() with concurrent use of nashorn
    • S8147468: Allow users to bound the size of buffers cached in the per-thread buffer caches
    • S8147645: get_ctrl_no_update() code is wrong
    • S8147807: crash in libkcms.so on linux-sparc
    • S8148379: jdk.nashorn.api.scripting spec. adjustments, clarifications
    • S8148627: RestrictTestMaxCachedBufferSize.java to 64-bit platforms
    • S8148820: Missing @since Javadoc tag in Logger.log(Level, Supplier)
    • S8148926: Call site profiling fails on braces-wrapped anonymous function
    • S8149017: Delayed provider selection broken in RSA client key exchange
    • S8149029: Secure validation of XML based digital signature always enabled when checking wrapping attacks
    • S8149330: Capacity of StringBuilder should not get close to Integer.MAX_VALUE unless necessary
    • S8149334: JSON.parse(JSON.stringify([])).push(10) creates an array containing two elements
    • S8149368: [hidpi] JLabel font is twice bigger than JTextArea font on Windows 7,HiDPI, Windows L&F
    • S8149411: PKCS12KeyStore cannot extract AES Secret Keys
    • S8149417: Use final restricted flag
    • S8149450: LdapCtx.processReturnCode() throwing Null Pointer Exception
    • S8149453: [hidpi] JFileChooser does not scale properly on Windows with HiDPI display and Windows L&F
    • S8149543: range check CastII nodes should not be split through Phi
    • S8149743: JVM crash after debugger hotswap with lambdas
    • S8149744: fix testng.jar delivery in Nashorn build.xml
    • S8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE
    • S8150002: Check for the validity of oop before printing it in verify_remembered_set
    • S8150470: JCK: api/xsl/conf/copy/copy19 test failure
    • S8150518: G1 GC crashes at G1CollectedHeap::do_collection_pause_at_safepoint(double)
    • S8150533: Test java/util/logging/LogManagerAppContextDeadlock.java times out intermittently.
    • S8150704: XALAN: ERROR: ‘No more DTM IDs are available’ when transforming with lots of temporary result trees
    • S8150780: Repeated offer and remove on ConcurrentLinkedQueue lead to an OutOfMemoryError
    • S8151064: com/sun/jdi/RedefineAddPrivateMethod.sh fails intermittently
    • S8151197: [TEST_BUG] Need to backport fix for test/javax/net/ssl/TLS/TestJSSE.java
    • S8151352: jdk/test/sample fails with “effective library path is outside the test suite”
    • S8151431: DateFormatSymbols triggers this.clone() in the constructor
    • S8151535: TESTBUG: java/lang/invoke/AccessControlTest.java should be modified to run with JTREG 4.1 b13
    • S8151731: Add new jtreg keywords to jdk 8
    • S8151998: VS2010 ThemeReader.cpp(758) : error C3861: ’round’: identifier not found
    • S8152927: Incorrect GPL header in StubFactoryDynamicBase.java reported
    • S8153252: SA: Hotspot build on Windows fails if make/closed folder does not exist
    • S8153531: Improve exception messaging for RSAClientKeyExchange
    • S8153641: assert(thread_state == _thread_in_native) failed: Assumed thread_in_native while heap dump
    • S8153673: [BACKOUT] JDWP: Memory Leak: GlobalRefs never deleted when processing invokeMethod command
    • S8154304: NullpointerException at LdapReferralException.getReferralContext
    • S8154722: Test gc/ergonomics/TestDynamicNumberOfGCThreads.java fails
    • S8157078: 8u102 L10n resource file updates
    • S8157838: Personalized Windows Font Size is not taken into account in Java8u102
  • Import of OpenJDK 8 u111 build 14
    • S6882559: new JEditorPane(“text/plain”,”") fails for null context class loader
    • S8049171: Additional tests for jarsigner’s warnings
    • S8063086: Math.pow yields different results upon repeated calls
    • S8140530: Creating a VolatileImage with size 0,0 results in no longer working g2d.drawString
    • S8142926: OutputAnalyzer’s shouldXXX() calls return this
    • S8147077: IllegalArgumentException thrown by api/java_awt/Component/FlipBufferStrategy/indexTGF_General
    • S8148127: IllegalArgumentException thrown by JCK test api/java_awt/Component/FlipBufferStrategy/indexTGF_General in opengl pipeline
    • S8150611: Security problem on sun.misc.resources.Messages*
    • S8153399: Constrain AppCDS behavior (back port)
    • S8157653: [Parfait] Uninitialised variable in awt_Font.cpp
    • S8158734: JEditorPane.createEditorKitForContentType throws NPE after 6882559
    • S8158994: Service Menu services
    • S8159684: (tz) Support tzdata2016f
    • S8160904: Typo in code from 8079718 fix : enableCustomValueHanlde
    • S8160934: isnan() is not available on older MSVC compilers
    • S8161141: correct bugId for JDK-8158994 fix push
    • S8162411: Service Menu services 2
    • S8162419: closed/com/oracle/jfr/runtime/TestVMInfoEvent.sh failing after JDK-8155968
    • S8162511: 8u111 L10n resource file updates
    • S8162792: Remove constraint DSA keySize < 1024 from jdk.jar.disabledAlgorithms in jdk8
    • S8164452: 8u111 L10n resource file update – msgdrop 20
    • S8165816: jarsigner -verify shows jar unsigned if it was signed with a weak algorithm
    • S8166381: Back out changes to the java.security file to not disable MD5
  • Backports
    • S8078628, PR3208: Zero build fails with pre-compiled headers disabled
    • S8141491, PR3159, G592292: Unaligned memory access in Bits.c
    • S8157306, PR3121: Random infrequent null pointer exceptions in javac (enabled on AArch64 only)
    • S8162384, PR3122: Performance regression: bimorphic inlining may be bypassed by type speculation
  • Bug fixes
    • PR3123: Some object files built without -fPIC on x86 only
    • PR3126: pax-mark-vm script calls “exit -1″ which is invalid in dash
    • PR3127, G590348: Only apply PaX markings by default on running PaX kernels
    • PR3199: Invalid nashorn URL
    • PR3201: Update infinality configure test
    • PR3218: PR3159 leads to build failure on clean tree
  • AArch64 port
    • S8131779, PR3220: AARCH64: add Montgomery multiply intrinsic
    • S8167200, PR3220: AArch64: Broken stack pointer adjustment in interpreter
    • S8167421, PR3220: AArch64: in one core system, fatal error: Illegal threadstate encountered
    • S8167595, PR3220: AArch64: SEGV in stub code cipherBlockChaining_decryptAESCrypt
    • S8168888, PR3220: Port 8160591: Improve internal array handling to AArch64.
  • Shenandoah
    • PR3224: Shenandoah broken when building without pre-compiled headers

The tarballs can be downloaded from:

We provide both gzip and xz tarballs, so that those who are able to make use of the smaller tarball produced by xz may do so.

The tarballs are accompanied by digital signatures available at:

These are produced using my public key. See details below.

  • PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
  • Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222

GnuPG >= 2.1 is required to be able to handle this key.

SHA256 checksums:

  • 88cc563d5cf4d7d0e8a394800ba580c922c5703dded4922551eb1a2425010b86 icedtea-3.2.0.tar.gz
  • 4cfd6876c99e5717b604e70460006e869ca77dea43fb97b3a697a2deb389b066 icedtea-3.2.0.tar.gz.sig
  • f2a197734cc1f820f14a6ba0aef0f198c24c77e9f026d14ddf185b684b178f80 icedtea-3.2.0.tar.xz
  • b92db947d9ba1b71c917bb16d7a312d1b01c1d99682a6b476c8302f9d2a981ae icedtea-3.2.0.tar.xz.sig

The checksums can be downloaded from:

A 3.2.0 ebuild for Gentoo is available.

The following people helped with these releases:

We would also like to thank the bug reporters and testers!

To get started:

$ tar xzf icedtea-3.2.0.tar.gz

or:

$ tar x -I xz -f icedtea-3.2.0.tar.xz

then:

$ mkdir icedtea-build
$ cd icedtea-build
$ ../icedtea-3.2.0/configure
$ make

Full build requirements and instructions are available in the INSTALL file.

Happy hacking!

This is a very important machine that really deserves to get built. Anyone who cares about Free Software should consider funding this project at some level, and spreading the word to their friends. If this project succeeds, it will bootstrap a market for new, owner-controlled performant desktop machines. If it fails, no such computers will exist. The project page and updates explain the current (rather depressing) state of general purpose computing better than I could, so take a look.

Valgrind 3.12.0 was just released with lots of exciting improvements. See the release notes for all the details. It is already packaged for Fedora 25.

Valgrind will also have a developer room at Fosdem on Saturday 4 February 2017 in Brussels, Belgium. Please join us, regardless of whether you are a Valgrind core hacker, Valgrind tool hacker, Valgrind user, Valgrind packager or hacker on a project that integrates, extends or complements Valgrind.

Please see the Call for Participation for more information on how to propose a talk or discussion topic.

I originally posted this on G+ but I thought maybe I should expand it a little and archive it here.

The patch to delete gcj went in recently.

When I was put on the gcj project at Cygnus, I remember thinking that Java was just a fad and that this was just a temporary thing for me. I wasn’t that interested in it. Then I ended up working on it for 10 years.

In some ways it was the high point of my career.

Socially it was fantastic, especially once we merged with the Classpath community — I’ve always considered Mark Wielaard’s leadership in that community as the thing that made it so great.  I worked with and met many great people while working on gcj and Classpath, but I especially wanted to mention Andrew Haley, who is the single best debugger I’ve ever met, and who stayed in the Java world, now working on OpenJDK.

We also did some cool technical things in gcj. The binary compatibility ABI was great, and the split verifier was very fun to come up with.  Per Bothner’s early vision for gcj drove us for quite a while, long after he left Cygnus and stopped working on it.

On the downside, gcj was never quite up to spec with Java. I’ve met Java developers even as recently as last year who harbor a grudge against gcj.

I don’t apologize for that, though. We were trying something difficult: to make a free Java with a relatively small team.

When OpenJDK came out, the Sun folks at FOSDEM were very nice to say that gcj had influenced the opening of the JDK. Now, I never truly believed this — I’m doubtful that Sun ever felt any heat from our ragtag operation — but it was very gracious of them to say so.

Since the gcj days I’ve been searching for basically the same combination that kept me hacking on gcj all those years: cool technology, great social environment, and a worthwhile mission.

This turned out to be harder than I expected. I’m still searching. I never thought it was possible to go back, though, and with this deletion, this is clearer than ever.

There’s a joy in deleting code (though in this case I didn’t get to do the deletion… grrr); but mainly this weekend I’m feeling sad about the final close of this chapter of my life.

The slides and video for my JavaOne 2016 talk about JDK 9 Language, Tooling, and Library Features have now been posted. There are many features in the release in addition to modules!

As an addendum to the parting message on coin-dev, one of the most popular candidate coin features not included in JDK 7 was some sort of literal syntax for collections, sets, lists, and/or maps. [1] [2]

Fortunately, this functionality is approximated in JDK 9 via Stuart Marks' JEP 269: Convenience Factory Methods for Collections which added factory methods named "of" that return immutable collections to the List, Set, and Map interfaces. I've enjoyed using this feature in my JDK 9 programming and suggest you give it a try too.

With that, I declare Project Coin well and fully minted! Happy coding.

PS I has heartened to see the improvements to numeric literals included by Project Coin in JDK 7 seem to have partially inspired the features set of an upcoming version of another widely-used language.

About a month ago, Fedora 24 has been released. This is an important milestone for Christine and myself, because it includes Shenandoah in its Java VM by default. We consider this our first official release!

If you want to try it out, it’s very simple: pass -XX:+UseShenandoahGC at the cmd line, and your favorite application runs with Shenandoah!

Please report back any issues that you find to the shenandoah-dev mailing list.


First, the underlying DataBasinKit framework got an important update.
[DBSoap update] now supports setting fields to null. That was quite a major detail missing: you could reset to blank even string fields.
This required me to fiddle a bit to generate the fieldsToNull list. Every field passed with an empty string value is considered to null.

<update xmlns="...">
<sobject xsi:type="sf:Account">
<id>....</id>
<fieldsToNull>AccountNumber</fieldsToNull>
<fieldsToNull>OtherCity</fieldsToNull>
<Name>New Name</Name>
</sobject>
</update>


The Object Inspector, the handy tool which allows you to inspect all field values of a record and knowing immediately their developer name given the Object Salesforce Id, how got update powers!



As the Screenshot (here on MacOS) shows, changed values show in a different color (non-updatable fields show in italics and their Cell is not editable). The total number of fields to be changed is summed up in the status field. Only fields marked as changed are updated when the Update button is pressed, other are left as-is and not overwritten for safety.

Further work has been done in the Inspector and full search filtering is now available!
Just entering a a sting will filter out the relevant rows. Both the Field Name or Developer Name are matched, as well as the content! It is thus super-easy to look for all fields (also fields not at layout) which have a certain Value. All fields false? easy as in the screenshot:



RoboVM 0.0.1 got released this week by Trillian AB.

RoboVM's main focus is to compile Java to native for deployment on mobile devices such as iOS and Android. RoboVM uses a Java to Objective-C bridge built using LLVM. Good news is that the same process work for converting Java applications to native applications on GNU/Linux systems as well!

Mario Zechner the author of libgdx posted this nice picture from inside DDD/GDB of his first HelloWorld compiled to native X86 code running on a GNU/Linux machine.
GNU/Linux machine code generated by RoboVM seen from inside DDD/GDB

http://www.robovm.org/

JogAmp is the home of high performance Java™ libraries for 3D Graphics, Multimedia and Processing.
JOGL, JOCL and JOAL provide cross platform Java™ language bindings to the OpenGL®, OpenCL™, OpenAL and OpenMAX APIs.
Running on Android, Linux, Window, OSX, and Solaris across devices using Java.

Release announcement for JogAmp 2.0.2-rc12

"You're encouraged to stop using the now-ancient 2.0-rc11!"

This 2.0.2-rc12 release include the largest security review in the 10-year history of JOGL

  • Security Fixes

    • Dynamic Linker Usage / Impl.
    • ProcAddressTable field visibility
    • Perform SecurityManager checks where required
    • Validation of property access
    • JAR Manifest tags:
      • Codebase
      • Permissions
      • Sealed
    • Use latest Java7 toolchain
      • Generating Java 1.6 bytecode
      • HTML API doc

https://jogamp.org/wiki/index.php/SW_Tracking_Report_Objectives_for_the_release_2.0.2_of_JOGL
Security fixes are marked in red on the above bug tracking page.
JogAmp send out thanks to the FuzzMyApp security researchers for healthy communication that triggered the security review work.

If you find an issue with the release, please report it to our bug database under the appropriate component. Development discussion takes place inside the JogAmp forum & mailing-list and the #jogamp IRC channel on irc.freenode.net.


Meet us @

JogAmp @ SIGGRAPH 2013

If you’ve been following Infinity and would like to, you know, download some code and try it out… well, now you can!

A jury found that using the declaring lines of code and their structure, sequence, and organization from Java constitutes fair use. Which is a great outcome of a terrible lawsuit Oracle filed some years ago trying to destroy free java. They started by trying to be a patent troll, but when that failed they tried to pervert copyright law to make it illegal to reimplement APIs. Oracle’s behavior is unethical and greedy. Luckily this jury stopped them for now.

screenshot

After more than a year of work, finally a new release of DataBasin and its associated framework. This version features a Framework that can be used in threads and the application itself is capable of having concurrent connection classes to SalesForce.com.

On Windows

On OpenBSD


Countless bug fixes (thanks to the bug reports of my colleagues Diego, Moustapha, Matteo + Matteo).

Major new features are:
  • threadable DataBasinKit framework
  • concurrent, interruptible operations (e.g. select vs. update)
  • handle multiple errors as a result of update
  • filter new lines when writing CSVs
  • countless bugfixes, especially in select-identify corner cases

DataBasinKit allows you native connection to SalesForce.com, allowing your application to integrate SOQL queries (SELECT, UPDATE similar operations), be it on Mac (Cocoa) or NetBSD, FreeBSD, OpenBSD, Linux or Solaris as well as Windows (MinGW).
DataBasin itself allows you to perform standard operations in a quick and agile interface: Extract accounts on Linux without the need of Java. Use the unique select-identify feature.

I am proud that Free Software can connect from a Free Software Operating System to a proprietary system and bridge the two worlds, enabling to do administrative work without being constrained to Java on Windows (read: DataLoader). Thanks to the many developers who continue supporting me in the development and keep these fine Operating Systems and tools alive.

Project Jigsaw is an enormous effort, encompassing six JEPs implemented by dozens of engineers over many years. So far we’ve defined a modular structure for the JDK (JEP 200), reorganized the source code according to that structure (JEP 201), and restructured the JDK and JRE run-time images to support modules (JEP 220). The last major component, the module system itself (JSR 376 and JEP 261), was integrated into JDK 9 earlier this week and is now available for testing in early-access build 111.

Breaking changes Like the previous major change, the introduction of modular run-time images, the introduction of the module system might impact you even if you don’t make direct use of it. That’s because the module system is now fully operative at both compile time and run time, at least for the modules comprising the JDK itself. Most of the JDK’s internal APIs are, as a consequence, fully encapsulated and hence, by default, inaccessible to code outside of the JDK.

An existing application that uses only standard Java SE APIs and runs on JDK 8 should just work, as they say, on JDK 9. If, however, your application uses a JDK-internal API, or uses a library or framework that does so, then it’s likely to fail. In many cases you can work around this via the -XaddExports option of the javac and java commands. If, e.g., your application uses the internal sun.security.x509.X500Name class then you can enable access to it via the option

-XaddExports:java.base/sun.security.x509=ALL-UNNAMED

This causes all members of the sun.security.x509 package in the java.base module to be exported to the special unnamed module in which classes from the class path are defined.

A few broadly-used internal APIs that cannot reasonably be implemented outside of the JDK, such as sun.misc.Unsafe, are still accessible for now. As outlined in JEP 260, however, these will be removed in a future release after suitable standard replacements are available.

The encapsulation of JDK-internal APIs is the change you’re most likely to notice when running an existing application. Other relevant but, for the most part, less-noticeable changes are described in the risks-and-assumptions section of JEP 261.

If you have trouble running an existing application on JDK 9 build 111 or later, and you think that’s due to the introduction of the module system but not caused by one of the changes listed in JEPs 260 or 261, then please let us know on the jigsaw-dev mailing list (you’ll need to subscribe first, if you haven’t already), or else submit a bug report via bugs.java.com.

New features If you’d like to start learning about the module system itself, the video of my Devoxx BE 2015 keynote gives a high-level overview and The State of the Module System summarizes the design of the module system proposed for JSR 376. Further details are available in the six Jigsaw JEPs, listed on the main project page, and in videos of other sessions given at JavaOne 2015 and Devoxx BE 2015.

The module-system design will continue to evolve in the JSR for a while yet, based on feedback and experience. The implementation will evolve in parallel in the Project Jigsaw “Jake” forest, and we’ll continue to publish bleeding-edge early-access builds based on that code, separately from the more-stable JDK 9 builds.

I finished getting Excorporate and all its dependencies into GNU ELPA. Excorporate lets Emacs retrieve calendar items from an Exchange server.

Excorporate in GNU ELPA

I had to rewrite the default UI to use Org Mode, because Calfw isn’t entirely copyright-assigned to the FSF yet. The Calfw UI is still there for reference, but as a text file so that GNU ELPA’s build and publishing steps ignore it. Both UI handlers use the same updated APIs from the main excorporate.el library.

Excorporate Org handler

I made sure Excorporate and all its dependencies use only features available since GNU Emacs 24.1. This is pretty good coverage; Emacs 24.1 introduced the packaging system, so if an Emacs version supports packages, it supports Excorporate.

Other than DNS lookups, Excorporate is completely asynchronous, so it won’t block the Emacs main loop. And it is pure Emacs Lisp so it runs on any operating system that Emacs does.

In addition to Org Mode support, release 0.7.0 collects all the suggestions users have made on this blog and adds Exchange 2007 support.

To install: M-x package-install RET excorporate

To get the source code:

git clone git://git.savannah.gnu.org/emacs/elpa.git

To report bugs: M-x report-emacs-bug

I wanted to attent FOSDEM two weeks ago, but couldn’t because I was sick, in bed with fever. I should have done a presentation about Shenandoah. Unfortunately, my backup Andrew Dinn also became sick that weekend, so that presentation simply didn’t happen. I want to summarize some interesting news that I wanted to show there. About Shenandoah’s performance.

When I talked about Shenandoah at FOSDEM 2015, I didn’t really announce any performance numbers, because we would have been embarrassed by them🙂 We spent the better part of last year optimizing the heck out of it, especially the barriers in the C2 compiler, and here we are, with some great results.

SPECjvm2015

Ok. This doesn’t really exist. The last SPECjvm release was SPECjvm2008. Unfortunately, SPEC doesn’t seem to care about SPECjvm anymore, which means the last Java version that runs it without any modifications is Java7. We did some small fixes, that allows it to run with Java9 too. This invalidates compliance of the results. But they are still tremendously useful for comparison. So here it comes:

SPECjvm2015

This was run on a 32 core box with 160GB of RAM, giving the JVM 140GB of heap. Exact same JVM and settings with G1 and Shenandoah. No special tuning parameters.

In terms of numbers, we get:

Throughput:   Shenandoah: 374 ops/m  vs. G1: 393 ops/m (95%, min 80%, max 140%)
Pauses:          Shenandoah: avg: 41 ms, max 202 ms                     G1:               avg: 240 ms, max 1126 ms

This means, throughput of Java apps, running with Shenandoah is on average 95% that of G1, depending on the actual application, it’ll range from around 80% to around 140%. However, pause times on such large heaps are significantly better with Shenandoah!

SPECjbb2015

SPECjbb2015 measures throughput of a simulated shop system under response time constraints, or service level agreements (SLAs). It measures ‘max-jops’ which is maximum throughput of the system without SLA, and critical-jops, which is throughput of the system under a restrictive SLA. Here are the numbers, G1 vs. Shenandoah, same machine and JVM settings as above:

SPECjbb2015This basically confirms the results that we got from SPECjvm2008: general throughput in the area of 95% behind G1, but much better pause times, reflected in a much higher score of critical-jops.

Other exciting news is that Shenandoah is now stable enough that we want to encourage everybody who’s interested to try it out. The nice folks at Adopt-OpenJDK have set up a nightly build from where you can grab binaries (Shenandoah JDK8 or Shenandoah JDK9). Enjoy! (And please report back if you encounter any problems!)



EclipseCon NA 2016

It was a great pleasure to have a chance to serve on this year’s EclipseCon Program Committee. As Java SE 8 adoption took place at “record-setting pace” during the past year, I was glad to see the EclipseCon team set their sights ahead, towards JDK 9, with its own track at EclipseCon. If you’d just like to take a peek at the changes being considered, developed and integrated into the JDK 9 Project, you can check out its web site in the OpenJDK community, and try out the Early Access builds.

If you’d like to hear what JDK 9 means for Eclipse, though, then you should come to EclipseCon in March and hear about it first hand from Jay Arthanareeswaran from IBM and Manoj Palat, who will talk about Java 9 support in Eclipse. In their session, they will look at what kind of support JDT provides for developers who would like to use JDK 9 in their projects, discussing planned Eclipse features as well as what modules could mean to different projects and how to best leverage the upcoming module system.

Within the OpenJDK community, Project Jigsaw is where development of the reference implementation of JSR 376 – Java Platform Module System takes place, along with the modularization of the JDK itself and the development of a new run-time image format. That’s a lot of new stuff to digest – fortunately, we’ll have Thomas Schindl at EclipseCon to give us a personal view and overview of what he calls "most likely the biggest change in Java’s history" in the “You, Me and Jigsaw” session.

At this point, you may be wondering if JDK 9 is all about modules. Modularity plays a huge role, but there is a lot more to it – more than 70 JDK Enhancement Proposals have been targeted for the JDK 9 release so far. To walk us through some of Java 9’s other puzzle pieces, we’ll have Erik Costlow from Oracle.

Finally, closing this track on Thursday, Erik will discuss “Preparing your code for JDK 9”. There are some steps you can take already to make your code ready to benefit from the new features planned for JDK 9, such as analyzing your project’s library dependencies for unintentional reliance on JDK-internal APIs.

I hope that you will enjoy this EclipseCon track, and that you will be inspired to start experimenting with JDK 9 and Eclipse.
At JavaOne, I'll be at the


OpenJDK Adoption Group BOF [BOF3377]

Monday, Oct 26, 8:00 p.m. | Hilton—Continental Ballroom 4


See you there!
Today I finally moved out of my lovely Ubuntu 10.04 LTS I was so proud in 2012, sad to read. That there are no longer any updates for it, only half of the problem. Many projects I am interested in started to fail during compilation because the toolchain is getting old. I was not updating because I did not like Unity, say whatever you want about it. I could learn it, of course, I have learned lots of way more sophisticated things (what about building GNU Classpath?). But I do not like the attitude, do not like decisions being made the way they are. Design must follow the user demands. Not in reverse. I gave a try to Gnome 3 as well with Fedora 22, played for some time but was not very impressed either and decided to drop after discovering that while it is possible to install the Gnome 2 - like Mate desktop, it is very difficult to actually switch into it.

My next distribution will be Ubuntu Mate. Value your freedom otherwise you will lose it!

Unexpectedly there were significant problems during updates . While in 2012 Linux installed no problem for me, now both Ubuntu Mate and Fedora 22 have booted in UEFI mode, raising lots of esoteric complains. It took half of the day for me to find where to switch of these UEFI beaties off in my wondercard, and another half to discover that the wondercard does not remember this setting, spontaneously reverting to "UEFI on". I cannot switch into UEFI, I have two other Linux distros on the same machine and do not want to loose. But now seems done.

This month I've released Orson PDF version 1.7, a compact and fast API for creating PDF content in Java through the standard Graphics2D API. This release features:

  • support for transparent images;
  • an implementation of the create() method to better support use against existing Java2D code;
  • addition of the GNU General Public License version 3 as the default license (a commercial license remains available for those that prefer it);
  • various bug fixes;
While Orson PDF has been created to provide PDF export for any Java2D-based code, my own use for it is within JFreeChart and Orson Charts. To provide an example, here is a chart that was exported with Orson PDF being viewed within Acrobat Reader: chartpdf.png

With the new GPLv3 license option, I've now also made the OrsonPDF repo at GitHub public, which will make it easier for other developers to work directly with the source code. You can also use GitHub to report any bugs or other issues.

The original version of this blog entry is published at http://www.object-refinery.com/blog/blog-20151008.html.

The first release candidate is finally available. It can be downloaded here or from NuGet.

What's New (relative to IKVM.NET 8.0):

  • Integrated OpenJDK 8u45.
  • Many fixes to late binding support.
  • Added ikvmc support for deterministic output files.
  • Various sun.misc.Unsafe improvements.
  • Many minor bug fixes and performance tweaks.

Changes since previous development snapshot:

  • Assemblies are strong named.
  • Fix for bug #303. ikvmc internal compiler error when trying to get interfaces from type from missing assembly reference.
  • Implemented NIO atomic file move on Windows.

Binaries available here: ikvmbin-8.1.5717.0.zip

Sources: ikvmsrc-8.1.5717.0.zip, openjdk-8u45-b14-stripped.zip

There is a nice profile of me on the Java Magazine of this bimestre, and I am very flattened for this so let me share it right away with you.

There is one question I was expecting though but didn’t come: “When did you start working on Java?”.

So, in order to give some more context, let me play with it and answer my own question here (and without space limits!). I think this is important, because it is about how I started to contribute to OpenJDK, it shows that you can do the same… if you are patient.

JM: When did you start working on Java?

Torre: I started to work in Java around its 1.3 release, and I used it ever since. I did start working on Java quite later though, around the Java 1.5/1.6 era probably. I was working to create an MSN messenger clone in Java on my Linux box, since all my friends where using it (MSN I mean, not Linux unfortunately), including the dreaded emoticons, and no Linux client supported those at the time.

I had all the protocol stuff working, I could handshake and share messages (although I still had to figure out the emoticons part!), but I had a terrible problem. I needed to save user credentials. Well, Java has a fantastic Preferences API, easy enough, right? Except that what I was using wasn’t the proprietary JDK, it was the Free Software version of it: GNU Classpath.

Classpath at the time didn’t have Preferences support, so I was stuck. I think somebody was writing a filesystem based preferences, or perhaps it was in Classpath but not GCJ, which is what everybody was using as a VM with the Classpath library, anyway when I started to look at the problem, I realised it would have been nicer to offer a GConf based Preferences store, and integrate the whole thing into the Gnome desktop (at the time, Gnome was a great desktop, nothing like today’s awfulness).

I was hooked. In fact, I even never finished my MSN messenger! After GConf, all sort of stuff came in, Decimal Formatter, GStreamer Sound backend, various fixes here and here, and this is when I learned a lot of how Swing works internally by following Sven de Marothy, Roman Kennke and David Gilbert work.

When Sun was about to release OpenJDK, I was in that very first group and witnessed the whole thing, a lot of behind the scenes of the creation of this extremely important code contribution. OpenJDK license is “GPL + Classpath exception” for a reason. I remember all the heroes that made Java Free Software.

I guess I was lucky, and the timing was perfect.

However right at the beginning contributing actual code to OpenJDK wasn’t at all easy like in Classpath. There was (is!) lot of process, things took a lot of time for anything but the most trivial changes etc…

But eventually I insisted and me and Roman where the first external guys to have code landing in the JDK, Roman was, I believe, the first independent person to have commit rights (I think that the people that are still today in my team at Red Hat and then also SAP had some changes already in, but at the time we two were the only guys completely external).

It wasn’t easy, I had to challenge ourselves and push a lot, and not give up. I had to challenge Sun, and even more challenge Oracle when it took the lead. But I did it. This is what I mean that everybody can do it, you can develop the skills and then you need to build the trust and then not let it go. I’m not sure what is more complex here, but if you persist it eventually come. And then all of a sudden billions of people will use your code and you are a Java Champion.

So this is how it started.


Final 8.1 development snapshot. Release candidate 0 will be next (after .NET 4.6 RTM).

Changes:

  • Updated HOWTO reference to OpenJDK 8u45.
  • Extract Windows version from kernel32.dll to avoid version lie. Idea stolen from OpenJDK.
  • Moved unused field removal optimization to a later stage in the compilation.
  • Made field removal optimization check more strict to only remove final fields and not remove fields that have annotations.
  • Added support for automatically passing in fields to "native" methods.
  • Various minor clean ups.
  • Added FieldWrapper.IsSerialVersionUID property to properly (and consistently) detect serialVersionUID fields.
  • Improved side effect free static initializer detection.
  • Improved -removeassertions ikvmc optimization to remove more code (esp. allow otherwise empty static initializers to be optimized away).

Binaries available here: ikvmbin-8.1.5666.zip

Just a couple of days ago I found out that some of my favourite musicians decided to join together to release an album, and allowed to preorder it on a crowdfunding website, Music Raiser.

The name of the band is “O.R.k.” and the founders are none but Lef, Colin Edwin, Pat Mastelotto and Carmelo Pipitone.

You probably have heard their names, if not, Colin Edwin is the bassist from Porcupine Tree while Carmelo Pipitone is the gifted guitarist from Marta Sui Tubi, an extremely original Italian band, they probably did the most interesting things in Italian music in the last 15 years or so; Lef, aka Lorenzo Esposito Fornasari, has done so many things that is quite hard to pick just one, but in Metal community he is probably best know for Obake. Finally, Pat Mastelotto is the drummer of King Crimson, and this alone made me jump on my seat!

One of the pre-order bonus was the ability to participate to a Remix Contest, and although I only got the stems yesterday in the late morning I could not resist to at least give it a try, and it’s a great honour for me that they have put my attempt on their Youtube channel:

It’s a weird feeling editing this music, after all, who am I to cut and remix and change the drum part (King Crimson, please forgive me!), how I ever dare to touch the guitars and voice, or rearrange the bass!?:)

But indeed it was a really fun experience, and I hope to be able do this again in the future.

And who knows, maybe they even like how I messed up their art and they decide to put me on their album! Nevertheless, it has been already a great honour for me to be able to see this material in semi-raw form (and a very interesting one!), so this has been already my first prize.

I’m looking forward now to listen the rest of the album!


I'm happy to announce that JFreeSVG version 3.0 has been uploaded to SourceForge. JFreeSVG is a fast and lightweight API for creating SVG content in Java. This release features:

  • new handling for BasicStroke cap, join and miterlimit;
  • a new ZIP option when writing SVG to files;
  • a demo for exporting Swing UIs to SVG;
  • removal of the CanvasGraphics2D implementation (to focus on SVG only);
  • a fix for handling of PathIterator.SEG_CLOSE;
  • a fix for y-coordinate bug in drawImage();
  • a workaround for ClassCastException when exporting Swing UIs on MacOSX with Nimbus L&F.

To ensure that JFreeSVG provides a fully functional Graphics2D implementation, I tested it using the Swingset3 demo with modifications to redirect the screen output directly to JFreeSVG to produce SVG output. I've always liked the way that Swing uses the Java2D API to cleanly separate its rendering from having any direct knowledge of the actual output target. Here is an example:

SVG not supported in your browser!

This turned out to be an effective test, because it uncovered a bug in one of the drawImage() methods that has remained undetected in all previous JFreeSVG releases.

One last thing...the JFreeSVG repo at GitHub is now public, which will make it easier for other developers to tweak the code for experimentation or bug fixes (if you spot a bug though, please report it to me).

If you'd like to give feedback on this post, please comment via the JFreeSVG forum.

I recently had occasion to scan some papers using a sheet-fed Ricoh printer/scanner/fax/copier. It seems to think that about 6 MB is as big of an email attachment as it can send so it splits up the PDFs into base64-encoded attachments. If you find yourself in a similar situation:

  • save the raw base64 text (if you’re using GMail, “show original” is your friend) and trim the extraneous text.
  • concatenate the multiple pieces together: cat part1 part2 > all.base64.
  • decode the whole thing: cat all.base64 | base64 -d > myscan.pdf.

As I wrote previously, Project Jigsaw is coming into JDK 9 in several large steps. JEP 200 defines the modular structure of the JDK, JEP 201 reorganizes the JDK source code into modular form, and JEP 220 restructures the JDK and JRE run-time images to support modules. The actual module system will be defined in JSR 376, which is just getting under way, and implemented by a corresponding JEP, yet to be submitted.

We implemented the source-code reorganization (JEP 201) last August. This step, by design, had no impact on developers or end users.

Most of the changes for modular run-time images (JEP 220) were integrated late last week and are now available in JDK 9 early-access build 41. This step, in contrast to the source-code reorganization, will have significant impact on developers and end users. All of the details are in the JEP, but here are the highlights:

  • JRE and JDK images now have identical structures. Previously a JDK image embedded the JRE in a jre subdirectory; now a JDK image is simply a run-time image that happens to contain the full set of development tools and other items historically found in the JDK.

  • User-editable configuration files previously located in the lib directory are now in the new conf directory. The files that remain in the lib directory are private implementation details of the run-time system, and should never be opened or modified.

  • The endorsed-standards override mechanism has been removed. Applications that rely upon this mechanism, either by setting the system property java.endorsed.dirs or by placing jar files into the lib/endorsed directory of a JRE, will not work. We expect to provide similar functionality later in JDK 9 in the form of upgradeable modules.

  • The extension mechanism has been removed. Applications that rely upon this mechanism, either by setting the system property java.ext.dirs or by placing jar files into the lib/ext directory of a JRE, will not work. In most cases, jar files that were previously installed as extensions can simply be placed at the front of the class path.

  • The internal files rt.jar, tools.jar, and dt.jar have been removed. The content of these files is now stored in a more efficient format in implementation-private files in the lib directory. Class and resource files previously in tools.jar and dt.jar are now always visible via the bootstrap or application class loaders in a JDK image.

  • A new, built-in NIO file-system provider can be used to access the class and resource files stored in a run-time image. Tools that previously read rt.jar and other internal jar files directly should be updated to use this file system.

We’re aware that these changes will break some applications, in particular IDEs and other development tools which rely upon the internal structure of the JDK. We think that the improvements to performance, security, and maintainability enabled by these changes are, however, more than worth it. We’ve already reached out to the maintainers of the major IDEs to make sure that they know about these changes, and we’re ready to assist as necessary.

If you have trouble running an existing application on JDK 9 build 41 or later and you think it’s due to this restructuring, yet not caused by one of the changes listed above or in JEP 220, then please let us know on the jigsaw-dev mailing list (you’ll need to subscribe first, if you haven’t already), or else submit a bug report via bugs.java.com. Thanks!