TEZ-4700: [Cloud] Single image shared between AM and Task

[Aggarwal-Raghav]

No description provided.

[Aggarwal-Raghav]

CC @abstractdog , This jira is more for discussion/organization of tez docker image.
Key changes:

1. Unified Docker Image: A single Dockerfile that builds an image for both AM and Task process.
2. Added a main entrypoint.sh that decide execution to either am-entrypoint.sh or child-entrypoint.sh based on the TEZ_COMPONENT environment variable.
3. The child-entrypoint.sh and am-entrypoint.sh are same except the main classname in java command. _I don't want to put everything into a main entrypoint.sh at this point. This decision can be taken at the end.
4. TEZ-4682 docker image i.e. apache/tez-am:1.0.0-SNAPSHOT -> apache/tez:1.0.0-SNAPSHOT. IMO we should publish the new one as part of TEZ-4698, will help the consumer to not switch to new one.

Points to discuss: (Its per my understanding and can be wrong)

1. As TEZ Child needs 5 java args: amHost, amPort, containerId, token, attemptNo. tez-am and tez-child are standalone docker process and tez-child will have to watch/read from zk /tez-external-sessions/tez_am/server to get this info. So, we would need to extend that capability like TEZ-4007 in TezChild as well, this is my understanding

2. The TaskScheduler and ContainerLauncher needs to be extended i.e. new implementation such that TezAM can start TezChild, TEZ-4665 needs to be completed before this. Once TEZ-4665 is done, we can get rid of tez.local.mode from docker tez-site.xml
3. Once TEZ-4665 + TEZ-4700 is done, TEZ-4663 will be automatically done. The ExternalAmWordCount.java should work.

[tez-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 7s		detect-secrets was not available.
+0 🆗	xmllint	0m 7s		xmllint was not available.
+0 🆗	shelldocs	0m 7s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+1 💚	mvninstall	10m 1s		master passed
+1 💚	compile	0m 41s		master passed
+1 💚	javadoc	0m 47s		master passed
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 21s		the patch passed
+1 💚	codespell	1m 20s		No new issues.
+1 💚	compile	0m 18s		the patch passed
+1 💚	javac	0m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	hadolint	0m 0s		No new issues.
+1 💚	markdownlint	0m 2s		No new issues.
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	yamllint	0m 1s		No new issues.
+1 💚	javadoc	0m 16s		the patch passed
			_ Other Tests _
+1 💚	unit	0m 18s		tez-dist in the patch passed.
+1 💚	asflicense	0m 15s		The patch does not generate ASF License warnings.
		17m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-472/2/artifact/out/Dockerfile
GITHUB PR	#472
Optional Tests	dupname asflicense javac javadoc unit codespell detsecrets xmllint compile shellcheck shelldocs yamllint hadolint markdownlint
uname	Linux 3468eed59e4f 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-agent/workspace/tez-multibranch_PR-472/src/.yetus/personality.sh
git revision	master / b4fff53
Default Java	Ubuntu-21.0.10+7-Ubuntu-124.04
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-472/2/testReport/
Max. process+thread count	74 (vs. ulimit of 5500)
modules	C: tez-dist U: tez-dist
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-472/2/console
versions	git=2.43.0 maven=3.8.7 hadolint=1.18.0-0-g76eee5c codespell=2.4.1 markdownlint=0.46.0 shellcheck=0.7.1 yamllint=1.38.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

[abstractdog]

I can see that a few PR are depending on each other now, like: TEZ-4700(this), TEZ-4665, TEZ-4698
I believe the ideal order to merge them would be:

1. TEZ-4700 (this)
2. TEZ-4698: given we have the common image name after this
3. TEZ-4665 -> and move every discussion to that place

so if we move this forward first, it doesn't have to contain any "child" specific content, because those can be added in TEZ-4665

does it make sense to you?

[Aggarwal-Raghav]

I can see that a few PR are depending on each other now, like: TEZ-4700(this), TEZ-4665, TEZ-4698 I believe the ideal order to merge them would be:

1. TEZ-4700 (this)
2. TEZ-4698: given we have the common image name after this
3. TEZ-4665 -> and move every discussion to that place

so if we move this forward first, it doesn't have to contain any "child" specific content, because those can be added in TEZ-4665

does it make sense to you?

ok, will remove child.env and child-entrypoint and update the docker-compose.yml etc.

[Aggarwal-Raghav]

@abstractdog , updated the PR

[tez-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	15m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+1 💚	mvninstall	4m 49s		master passed
+1 💚	compile	0m 25s		master passed
+1 💚	javadoc	0m 27s		master passed
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 12s		the patch passed
+1 💚	codespell	0m 58s		No new issues.
+1 💚	compile	0m 11s		the patch passed
+1 💚	javac	0m 11s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	hadolint	0m 0s		No new issues.
+1 💚	markdownlint	0m 3s		No new issues.
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	yamllint	0m 0s		No new issues.
+1 💚	javadoc	0m 10s		the patch passed
			_ Other Tests _
+1 💚	unit	0m 11s		tez-dist in the patch passed.
+1 💚	asflicense	0m 11s		The patch does not generate ASF License warnings.
		24m 14s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-472/3/artifact/out/Dockerfile
GITHUB PR	#472
Optional Tests	dupname asflicense javac javadoc unit codespell detsecrets xmllint compile shellcheck shelldocs yamllint hadolint markdownlint
uname	Linux cce67f8a833c 5.15.0-174-generic #184-Ubuntu SMP Fri Mar 13 18:41:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-agent/workspace/tez-multibranch_PR-472/src/.yetus/personality.sh
git revision	master / 75b0506
Default Java	Ubuntu-21.0.10+7-Ubuntu-124.04
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-472/3/testReport/
Max. process+thread count	81 (vs. ulimit of 5500)
modules	C: tez-dist U: tez-dist
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-472/3/console
versions	git=2.43.0 maven=3.9.14 hadolint=1.18.0-0-g76eee5c codespell=2.4.1 markdownlint=0.46.0 shellcheck=0.7.1 yamllint=1.38.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.