Vector API in Java (Incubator)

Imagine a teacher checking answer sheets. One teacher can mark one sheet at a time. Four teachers sitting side by side, each with the same answer key, can mark four sheets in the time it takes to mark one. Modern CPUs have something like that built in: special instructions that do the same operation on several numbers at once. A normal Java loop usually adds one pair of numbers per step. The Vector API lets you ask for four, eight or more at a time. In this lesson, you will learn what the Vector API is, its main ideas, and how to write a loop with it.

What Is the Vector API?

The Vector API in Java is a library in the module jdk.incubator.vector that is used to write vector computations: one operation applied to a whole group of values in a single step. On supported hardware, the JIT compiler turns these operations into the CPU's SIMD instructions (Single Instruction, Multiple Data), such as AVX on Intel and AMD, or NEON and SVE on ARM.

Key Ideas

The best species depends on the CPU. SPECIES_PREFERRED picks the widest one the current machine handles well, so the same code uses 4 lanes on one computer and 8 or 16 on another.

Main Classes and Methods

Adding Two Arrays

Let's see an example that adds two arrays of marks lane by lane.

import java.util.Arrays;
import jdk.incubator.vector.IntVector;
import jdk.incubator.vector.VectorSpecies;
 
public class AddMarks {
    static final VectorSpecies<Integer> SPECIES = IntVector.SPECIES_PREFERRED;
 
    public static void main(String[] args) {
        int[] theory    = {62, 71, 55, 80, 67, 74, 59, 88, 70, 65};
        int[] practical = {20, 18, 25, 19, 22, 24, 21, 17, 23, 25};
        int[] total = new int[theory.length];
 
        int i = 0;
        int bound = SPECIES.loopBound(theory.length);
        for (; i < bound; i += SPECIES.length()) {
            IntVector a = IntVector.fromArray(SPECIES, theory, i);
            IntVector b = IntVector.fromArray(SPECIES, practical, i);
            a.add(b).intoArray(total, i);
        }
        for (; i < theory.length; i++) {        // the leftover elements
            total[i] = theory[i] + practical[i];
        }
        System.out.println(Arrays.toString(total));
    }
}

[82, 89, 80, 99, 89, 98, 80, 105, 93, 90]

Explanation:

1. SPECIES_PREFERRED chooses the lane count for this CPU. On the computer used for this lesson (Apple Silicon with NEON), it is 4 ints.
2. loopBound(10) returns 8 when there are 4 lanes, so the vector loop covers elements 0 to 7 in two steps.
3. Each step loads four theory marks and four practical marks, adds all four pairs at once with add, and stores them with intoArray.
4. The second, ordinary loop handles the tail: elements 8 and 9, which don't fill a whole vector.
5. On a CPU with 8 lanes, the same code would do one vector step and a tail of 2. The result is the same either way.

Summing with a Reduction

A reduction combines lanes into one value. The usual pattern keeps a running vector of partial sums and reduces it once at the end.

import jdk.incubator.vector.FloatVector;
import jdk.incubator.vector.VectorOperators;
import jdk.incubator.vector.VectorSpecies;
 
public class FeeTotal {
    static final VectorSpecies<Float> SPECIES = FloatVector.SPECIES_PREFERRED;
 
    static float sum(float[] values) {
        FloatVector partial = FloatVector.zero(SPECIES);
        int i = 0;
        int bound = SPECIES.loopBound(values.length);
        for (; i < bound; i += SPECIES.length()) {
            partial = partial.add(FloatVector.fromArray(SPECIES, values, i));
        }
        float total = partial.reduceLanes(VectorOperators.ADD);
        for (; i < values.length; i++) {
            total += values[i];
        }
        return total;
    }
 
    public static void main(String[] args) {
        float[] fees = new float[1000];
        java.util.Arrays.fill(fees, 1500);
        System.out.println(sum(fees));
        System.out.println(sum(new float[] {1, 2, 3}));
    }
}

1500000.0
6.0

Explanation: partial holds one running total per lane. After the loop, reduceLanes(ADD) adds the lanes together, and the tail loop adds whatever is left. The second call shows that a tiny array, shorter than one vector, is handled entirely by the tail loop.

On the computer used for this lesson, summing a million floats this way took about 0.14 ms instead of 0.48 ms for a plain loop, roughly 3.5 times faster with 4 lanes. Wider CPUs gain more.

When It Helps

The JIT already auto-vectorizes many simple loops. A plain float sum is a common exception: Java must add floats in the exact order you wrote, so the JIT can't reorder them into lanes. The Vector API lets you say that a different order is fine.

Common Mistakes

Forgetting the tail loop. If the array length isn't a multiple of the lane count, the last elements are silently skipped.

Hard-coding the lane count. Writing i += 4 breaks on CPUs with 8 lanes. Always use SPECIES.length().

Expecting identical float results. A vector sum adds in a different order, so the last digits of a float result can differ slightly from a plain loop.

Forgetting --add-modules jdk.incubator.vector. Without it, the compiler reports that the package isn't visible.

Practice

1. Easy: Change AddMarks to multiply each theory mark by 2 with mul before adding. Check the first result by hand.
2. Harder: Write max(int[] values) using IntVector and reduceLanes(VectorOperators.MAX), with a tail loop. Test it on an array of 13 marks.

Key Takeaways