How do I build the Android SDK with hidden and internal APIs available?

I want to rebuild the Android SDK (or rather only the android.jar) to include hidden and internal APIs.

I could not find any documentation or discussion doing on how to go about this. I have an Ubuntu CyanogenMod build environment already setup that is able to build cm7.

Now, I read that make SDK will build the SDK but I want to build an SDK that includes methods and fields that are marked as hidden using @hide. Is this possible?

What I want to do is make changes to an application that uses hidden API and in order to rebuild it I would like to use the modified SDK.

This is what I always do to use hidden api.

1. Build the repo or download jars from https://sites.google.com/site/hippunosource/home/android/androidnohide-apiwo-shi-yongsuru-rifurekushonha-wei-shi-yong
2. copy out out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar (better to rename it as something like framework_all.jar)
3. config your project build path-->libraries --> add this external jars. In Order and Export, move it up and before android.jar

I have done some investigating into this, and my conclusion is simply: This cannot be done without quite a bit of work. Read the rest of this answer for details on what I have found.

---

android.jar is actually comprised of the "public api" of framework.jar and core.jar which is found in system/frameworks/ on the device. android.jar is a kind of what I would call Java library header, all implementation in the actual byte code are just a throw new RuntimeException("stub");, this allows you to build against android.jar (e.g. in Eclipse), but execution has to be performed on a device or emulator.

The public API of the Android SDK is defined by classes/methods/fields that are not prefixed with the @{hide} javadoc annotation. I.e. everything that is not annotated is included in the SDK.

android.jar is built from the sources located in out/target/common/obj/JAVA_LIBRARIES/android_stubs_current_intermediates which itself is generated by the tool DroidDoc located in build/tools/droiddoc.

DroidDoc is the tool (probably adapted from javadoc, or using javadoc) that generate the actual Android SDK documentation. As a side-effect, and probably because it is already parsing all the javadoc, it also spews out the android stubs which are then compiled into the android.jar which is distributed in the SDK.

So to include the stuff that is hidden you could, if you only want to include specific parts, just remove the @hide annotation and rebuild the SDK.

However if you want to include all the hidden parts things get a lot more complicated. You can modify DroidDoc (the relevant source is in build/tools/droiddoc/src/Stubs.java) such that nothing is detected as hidden. This is quite trivial and I have tried this, however the stubs that is then generated does not compile at all.

My conclusion by now is that this is simply not feasible. The stubs generated if you remove the part of DroidDoc that detect hidden annotations, is simply not compilable, and would require quite a bit of work to compile correctly.

So my answer to your questions is: No, this cannot be done, without doing a lot of work. Sorry.

---

A side note about the mkstubs tool. mkstubs are used when you build a SDK addon, i.e. the addons you can find in the Android SDK manager from vendors, e.g. Samsung providing you with an additional API for stuff specific to Samsung phones. mkstubs does much the same as the DroidDoc stubs generation process, however it does not use @hide annotations, it uses a .defs file describing which packages/classes/fields to include or exclude from your SDK addon.

However this is all irrelevant to the question, as the Android SDK build does not use the mkstubs tool. (Unfortunately.)

We could reconstruct the *.jar files from the Android platform.

First, connect ADB to your device. Then run:

adb pull /system/framework/core.jar .
adb pull /system/framework/framework.jar .

The core.jar contain the standard Java libraries (java.*) and the framework.jar contain the Android libraries (android.*). This is not usable yet, as the actual files are in DEX format, not JAR format.

We could convert these DEX-formatted *.jars into real JARs using tools such as dex2jar:

dex2jar core.jar
dex2jar framework.jar

Then pull in these jars using "Add External JARs..." (assuming you're using Eclipse ADT)

• right click on Project → Properties → Java Build Path → Libraries → Add External JARs... → (Choose the core-dex2jar.jar and framework-dex2jar.jar from above).

This will enable you to use the internal and some Java 7 APIs. (The generated APK, as far as I can see, does not contain any actual code from the JARs.)

You can download the modified android.jar to be used as hidden APIs from this repository. Follow the instructions there.

For Lollipop the flow is little different:

0. Get /system/framework/arm/boot.oat from lollipop device

1. Use 'java -jar oat2dex.jar boot boot.oat'

2. You will get two folders: dex and odex. Go to dex and make 'java -jar dex2jar.jar framework.dex'

3. Rename resulting framework.jar to .zip, extract and find classes you need

4. Go to [sdk_path]/platforms/[target_platform] and extract android.jar (first rename it to zip).

5. Copy files from extracted framework to extracted android.jar. Then compress to zip and rename to .jar :)

ps: probably you need repeat steps 4-6 for 'framework_classes2.dex'

DroidCon 2011

Here Erik Hellman from Sony Ericson explains how to access the hidden Android API's:

http://vimeo.com/30180393 (Hmm link doesn't appear to work).

Goto the DroidCon webpage Day 2 scroll down to Using Hidden APIs 10:15 and you can watch it there.

Links are dieing!

I've found this one: http://skillsmatter.com/podcast/os-mobile-server/hidden-api I don't know, how long it'll be up.

> The official APIs in the Android SDK is usually sufficient for most normal applications. However, there are sometimes situations where a developer needs access to the internal system services, APIs and resources that are not published in the official APIs. Fortunately, these APIs are still available through some clever tricks and can often be useful when developing new and innovative solution on top of Android. In this session you will learn how to access and use these hidden and protected APIs, the limitations of their usage and some tips'n'trick on how to use them in a safe and control manner across multiple vendors devices and Android versions. The audience will see several advanced demos that you normally cannot do with Android. Expect a fairly advanced session with lots of insights in the internals of the Android platform.

Try to look at this: > The ultimate target of these articles is to give developers the power of Internal and Hidden APIs without using reflection. If you complete all the steps described in next several parts you will be able to use Internal and Hidden APIs as if they were public open APIs. There will be no need for reflection. > But if you’re using these non-public APIs then you should be aware that your application is at great risk. Basically there are no guarantees that APIs will not be broken with next update to Android OS. There are even no guarantees about consistent behavior across devices from different vendors. You are completely on your own. > There are three scenarios you may want to follow: >

1. Enable both internal and hidden APIs (scenario A)
2. Enable only hidden API (scenario B)
3. Enable only internal API (scenario C) > Scenario A is a sum of B and C. Scenario B is the easiest one (requires no eclipse ADT plugin modifications). > Scenario A: read parts 1, 2, 3, 4, 5 > Scenario B: read parts 1, 2, 3, 5 > Scenario C: read parts 1, 2, 3, 4, 5

I once wrote some Groovy scripts for extracting the java files from a repo checkout from http://source.android.com/ and then compiling them without the need for a full toolchain for compiling all the android sources, including the needed other steps (packaging, generating resources etc).

They can be found here:

https://github.com/thoutbeckers/CollectAndroid

But for sure this will need updating for anything after Gingerbread, mostly by setting the correct directories in "rootdirs" in the config file (CollectConfig.groovy).

At the time I regularly used this for development with all of the hidden API and sources (also problematic at the time) available.

As mentioned elsewhere com/android/internal/** will still be hidden in recent versions of ADT due to the access rule aded.

Long's answer worked for me, but I was still missing some classes I needed, in particular android.provider.Telephony. I was able to add it like this:

1. Extract the framework.jar file

   mkdir /tmp/framework
   cp framework.jar /tmp
   cd /tmp/framework
   jar xvf ../framework.jar
   mv android classes
   

2. Build the Android repo, which will create the out/target/common/obj/JAVA_LIBRARIES directory

3. Find where the missing classes are

   $ cd /path/to/out/target/common/obj/JAVA_LIBRARIES
   $ find . | grep "/Telephony.class"
   ./telephony-common_intermediates/classes/android/provider/Telephony.class
   ./android_stubs_current_intermediates/classes/android/provider/Telephony.class
   

4. Add the new classes and rebuild the framework JAR file

   cd /tmp/framework
   cp -r /path/to/out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes .
   cp -r /path/to/out/target/common/obj/JAVA_LIBRARIES/telephony-common_intermediates/classes .
   cd classes
   jar cvf ../framework.jar .
   

Or you can just be lazy and include all of the classes into one giant jar file:

cd /tmp/framework
cp -r /path/to/out/target/common/obj/JAVA_LIBRARIES/*/classes .
cd classes
jar cvf ../framework.jar .

I can't comment but this is basically a comment to @KennyTM's (https://stackoverflow.com/a/13550030/2923406) excellent answer:

If you find yourself with the following error in Eclipse:

The type com.android.internal.util.Predicate cannot be resolved. It is indirectly referenced from required .class   files

(that is, android.internal.* is not available)

Then one possible solution is to apply the same method for /system/framework/framework2.jar. Using the Android Emulator for SDK19 I have this extra jar. On my HTC One there is even a framework3.jar.