foldMap in Java Stream API?

[jovial elbow]

I understand the concept of flatMap in functional programming kind of, but I am totally lost about it Java. For example:

public class TransformationsWithFlatMap {

    private static final List<List<String>> arrayListOfNames = List.of(
            List.of("Mariam", "Alex", "Ismail"),
            List.of("John", "Alesha", "Andre"),
            List.of("Susy", "Ali")
    );

    public void withFlatMap() throws Exception {
        // [Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
        Function<List<String>, Stream<? extends String>> stream = List::stream;
        List<String> result = arrayListOfNames.stream().flatMap(stream).collect(Collectors.toList());
    }
}

First of all, flatMap seem to expect ( java.util.function.Function<? super T, ? extends Stream<? extends R>> mapper ) when we use it for anything. I have no idea what that means.

And then if I insert List::stream (according to some tutorial) that is equal to Function<List<String>, Stream<? extends String>>. This seem like chinese to me.

How can I understand flatMap in Java streams better, and be able to read complex types like the one shown here?

[snow oriole]

also stream is very inefficient and consume lots of temporary objects and ram/cpu 🤣

.stream() works PERFECTLY on large FLAT FILES, where you cannot slurp the entire file into RAM
say 300 MiB needs over 32 GiB after all the String / char[] copies

;compile java

import java.util.*;
import java.util.function.*;
import java.util.stream.*;

public class TransformationsWithFlatMap
{
    private static final List<List<String>> arrayListOfNames = List.of(
            List.of("Mariam", "Alex", "Ismail"),
            List.of("John", "Alesha", "Andre"),
            List.of("Susy", "Ali")
    );

    public static void main(String[] args)
    throws Exception
    {
        // [Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
System.out.println(new Date().getTime());
        final Function<List<String>, Stream<? extends String>> stream = List::stream;
        final List<String> result = arrayListOfNames.stream().flatMap(stream).collect(Collectors.toList());
        System.out.println(result);
System.out.println(new Date().getTime());
    }
}

[shrewd rain]

Program Output

1692057008916
[Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
1692057008928

fd26#0000 | 140ms | java | jdk 20.0.0 | godbolt.org

[snow oriole]

so this one flatenize a 2D array into 1D array

https://github.com/AdoptOpenJDK/openjdk-jdk12u/blob/master/src/java.base/share/classes/java/util/List.java

https://github.com/AdoptOpenJDK/openjdk-jdk12u/blob/master/src/java.base/share/classes/java/util/Collection.java

/**
     * Returns a sequential {@code Stream} with this collection as its source.
     *
     * <p>This method should be overridden when the {@link #spliterator()}
     * method cannot return a spliterator that is {@code IMMUTABLE},
     * {@code CONCURRENT}, or <em>late-binding</em>. (See {@link #spliterator()}
     * for details.)
     *
     * @implSpec
     * The default implementation creates a sequential {@code Stream} from the
     * collection's {@code Spliterator}.
     *
     * @return a sequential {@code Stream} over the elements in this collection
     * @since 1.8
     */
    default Stream<E> stream() {
        return StreamSupport.stream(spliterator(), false);
    }

so List::stream() is just Collection::stream()

 /**
     * Creates a new sequential or parallel {@code Stream} from a
     * {@code Spliterator}.
     *
     * <p>The spliterator is only traversed, split, or queried for estimated
     * size after the terminal operation of the stream pipeline commences.
     *
     * <p>It is strongly recommended the spliterator report a characteristic of
     * {@code IMMUTABLE} or {@code CONCURRENT}, or be
     * <a href="../Spliterator.html#binding">late-binding</a>.  Otherwise,
     * {@link #stream(java.util.function.Supplier, int, boolean)} should be used
     * to reduce the scope of potential interference with the source.  See
     * <a href="package-summary.html#NonInterference">Non-Interference</a> for
     * more details.
     *
     * @param <T> the type of stream elements
     * @param spliterator a {@code Spliterator} describing the stream elements
     * @param parallel if {@code true} then the returned stream is a parallel
     *        stream; if {@code false} the returned stream is a sequential
     *        stream.
     * @return a new sequential or parallel {@code Stream}
     */
    public static <T> Stream<T> stream(Spliterator<T> spliterator, boolean parallel) {
        Objects.requireNonNull(spliterator);
        return new ReferencePipeline.Head<>(spliterator,
                                            StreamOpFlag.fromCharacteristics(spliterator),
                                            parallel);
    }

so you are going to create :

• Stream
• ReferencePipeline.Head<>
• spliterator
• StreamOpFlag

https://github.com/AdoptOpenJDK/openjdk-jdk12u/blob/master/src/java.base/share/classes/java/util/stream/ReferencePipeline.java#L617

which is going to super

• ReferencePipeline
• AbstractPipeline
• Supplier

which is going to create up to 3 AbstractPipeline for each stage*

https://github.com/AdoptOpenJDK/openjdk-jdk12u/blob/master/src/java.base/share/classes/java/util/stream/AbstractPipeline.java

then you are going to create a

• Collector
• BiConsumer
• ReduceOps
• ForEachTask or ForEachOrderedTask

;compile java

import java.util.*;
import java.util.function.*;
import java.util.stream.*;

public class TransformationsWithFlatMap
{
    private static final List<List<String>> arrayListOfNames = List.of(
            List.of("Mariam", "Alex", "Ismail"),
            List.of("John", "Alesha", "Andre"),
            List.of("Susy", "Ali")
    );

    public static void main(String[] args)
    throws Exception
    {
        // [Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
System.out.println(new Date().getTime());
        final ArrayList<String> result = new ArrayList<String>(64);
        for(final List<String> inner : arrayListOfNames)
        {
          for(final String name : inner)
          {
            result.add(name);
          }
        }
        System.out.println(result);
System.out.println(new Date().getTime());
    }
}

[shrewd rain]

Program Output

1692057098228
[Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
1692057098229

fd26#0000 | 93ms | java | jdk 20.0.0 | godbolt.org

[snow oriole]

;compile java

import java.util.*;
import java.util.function.*;
import java.util.stream.*;

public class TransformationsWithFlatMap
{
    private static final List<List<String>> arrayListOfNames = List.of(
            List.of("Mariam", "Alex", "Ismail"),
            List.of("John", "Alesha", "Andre"),
            List.of("Susy", "Ali")
    );

    public static void main(String[] args)
    throws Exception
    {
        // [Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
System.out.println(new Date().getTime());
        final ArrayList<String> result = new ArrayList<String>(64);
        for(final List<String> inner : arrayListOfNames)
        {
          for(final String name : inner)
          {
            result.add(name);
          }
        }
        result.trimToSize();
        System.out.println(result);
System.out.println(new Date().getTime());
    }
}

[shrewd rain]

Program Output

1692057144413
[Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
1692057144415

fd26#0000 | 157ms | java | jdk 20.0.0 | godbolt.org

[snow oriole]

so 1-2 ms versus 12 ms for a "small double array"

also which one is easier to read / maintain / adjust / debug ? 🤔

;compile java

import java.util.*;
import java.util.function.*;
import java.util.stream.*;

public class TransformationsWithFlatMap
{
    private static final List<List<String>> arrayListOfNames = List.of(
            List.of("Mariam", "Alex", "Ismail"),
            List.of("John", "Alesha", "Andre"),
            List.of("Susy", "Ali")
    );

    public static void main(String[] args)
    throws Exception
    {
        // [Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
System.out.println(new Date().getTime());
        final ArrayList<String> result = new ArrayList<String>(64);
        for(final List<String> inner : arrayListOfNames)
        {
          for(final String name : inner)
          {
            result.add(name);
          }
        }
        result.trimToSize();
        final List<String> r = Collections.unmodifiableList(result);
        System.out.println(r);
System.out.println(new Date().getTime());
    }
}

[shrewd rain]

Program Output

1692057431972
[Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
1692057431973

fd26#0000 | 125ms | java | jdk 20.0.0 | godbolt.org

[snow oriole]

;compile java

import java.util.*;
import java.util.function.*;
import java.util.stream.*;

public class TransformationsRefactored
{
    private static final List<List<String>> arrayListOfNames = List.of(
            List.of("Mariam", "Alex", "Ismail"),
            List.of("John", "Alesha", "Andre"),
            List.of("Susy", "Ali")
    );

    public static <T> List<T> transformToFlatList(final Collection<? extends Collection<T> > list)
    {
        final int n = list.size();
        final int size = Math.max(16, n * n);
        final ArrayList<T> result = new ArrayList<T>(size);
        for(final Collection<T> inner : list)
        {
          for(final T name : inner)
          {
            result.add(name);
          }
        }
        result.trimToSize();
        final List<T> r = Collections.unmodifiableList(result);
        return r;
    }

    public static void main(String[] args)
    throws Exception
    {
        // [Mariam, Alex, Ismail, John, Alesha, Andre, Susy, Ali]
System.out.println(new Date().getTime());

        final List<String> r = transformToFlatList(arrayListOfNames);
        System.out.println(r);

System.out.println(new Date().getTime());
    }
}