{"id":17538,"date":"2023-09-15T08:04:45","date_gmt":"2023-09-15T01:04:45","guid":{"rendered":"https:\/\/www.thegioimaychu.vn\/blog\/?p=17538"},"modified":"2023-09-15T08:11:05","modified_gmt":"2023-09-15T01:11:05","slug":"apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark","status":"publish","type":"post","link":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/","title":{"rendered":"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark"},"content":{"rendered":"<style>.iblink{color:#975!important;}.iblink:hover{color:orange!important;}<\/style><p><a href=\"https:\/\/spark.apache.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">Apache Spark<\/a> l\u00e0 m\u1ed9t framework m\u00e3 ngu\u1ed3n m\u1edf \u0111\u1ec3 x\u1eed l\u00fd song song c\u00e1c t\u00e1c v\u1ee5 d\u1eef li\u1ec7u l\u1edbn tr\u00ean c\u00e1c h\u1ec7 th\u1ed1ng \u0111\u01b0\u1ee3c ph\u00e2n c\u1ee5m.\u00a0\u0110\u00e2y l\u00e0 m\u1ed9t trong nh\u1eefng framework x\u1eed l\u00fd ph\u00e2n t\u00e1n \u0111\u01b0\u1ee3c s\u1eed d\u1ee5ng r\u1ed9ng r\u00e3i nh\u1ea5t tr\u00ean th\u1ebf gi\u1edbi.<\/p>\n<h2>Apache Spark l\u00e0 g\u00ec?<\/h2>\n<p>Song song v\u1edbi s\u1ef1 gia t\u0103ng v\u01b0\u1ee3t b\u1eadc c\u1ee7a d\u1eef li\u1ec7u ng\u00e0y nay tr\u00ean t\u1ea5t c\u1ea3 c\u00e1c l\u0129nh v\u1ef1c, Apache Spark \u0111\u00e3 tr\u1edf th\u00e0nh m\u1ed9t trong nh\u1eefng framework ph\u1ed5 bi\u1ebfn nh\u1ea5t \u0111\u1ec3 x\u1eed l\u00fd d\u1eef li\u1ec7u ph\u00e2n t\u00e1n quy m\u00f4 l\u1edbn, ch\u1ea1y tr\u00ean h\u00e0ng tri\u1ec7u m\u00e1y ch\u1ee7, c\u1ea3 t\u1ea1i ch\u1ed7 v\u00e0 tr\u00ean \u0111\u00e1m m\u00e2y.<\/p>\r\n\t\t\t\t<div class=\"tgmcad hor\" style=\"text-align:center;margin:30px 0; padding: 10px 0; border-radius:5px; border-top:solid 1px #ccc; border-bottom:solid 1px #ccc; \">\r\n\t\t\t\t<a href=\"https:\/\/thegioimaychu.vn\/giai-phap\/deep-learning\/\"><img decoding=\"async\" alt=\"Gi\u1ea3i ph\u00e1p h\u1ea1 t\u1ea7ng Deep Learning, Tr\u00ed tu\u1ec7 Nh\u00e2n t\u1ea1o - AI\" src=\"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2019\/11\/tgmc-deep-learning.jpg\" \/><\/a>\r\n\t\t\t\t<\/div>\n<p>Apache Spark l\u00e0 m\u1ed9t c\u00f4ng c\u1ee5 ph\u00e2n t\u00edch nhanh, \u0111a n\u0103ng \u0111\u1ec3 x\u1eed l\u00fd d\u1eef li\u1ec7u quy m\u00f4 l\u1edbn ch\u1ea1y tr\u00ean YARN, Apache Mesos, Kubernetes, n\u1eb1m \u0111\u1ed9c l\u1eadp ho\u1eb7c tr\u00ean d\u1ecbch v\u1ee5 \u0111\u00e1m m\u00e2y.\u00a0V\u1edbi c\u00e1c to\u00e1n t\u1eed v\u00e0 th\u01b0 vi\u1ec7n c\u1ea5p cao d\u00e0nh cho SQL, x\u1eed l\u00fd lu\u1ed3ng (stream processing), h\u1ecdc m\u00e1y v\u00e0 x\u1eed l\u00fd \u0111\u1ed3 th\u1ecb, Spark gi\u00fap d\u1ec5 d\u00e0ng x\u00e2y d\u1ef1ng c\u00e1c \u1ee9ng d\u1ee5ng song song trong Scala, Python, R ho\u1eb7c SQL b\u1eb1ng c\u00e1ch s\u1eed d\u1ee5ng interactive shell, notebook ho\u1eb7c \u1ee9ng d\u1ee5ng \u0111\u00f3ng g\u00f3i (packaged applications).\u00a0Spark h\u1ed7 tr\u1ee3 ph\u00e2n t\u00edch theo batch v\u00e0 t\u01b0\u01a1ng t\u00e1c b\u1eb1ng c\u00e1ch s\u1eed d\u1ee5ng m\u00f4 h\u00ecnh l\u1eadp tr\u00ecnh ch\u1ee9c n\u0103ng v\u00e0 c\u00f4ng c\u1ee5 truy v\u1ea5n li\u00ean quan Catalyst, chuy\u1ec3n \u0111\u1ed5i c\u00f4ng vi\u1ec7c th\u00e0nh k\u1ebf ho\u1ea1ch v\u00e0 l\u1ecbch ho\u1ea1t \u0111\u1ed9ng b\u00ean trong truy v\u1ea5n tr\u00ean c\u00e1c node trong m\u1ed9t c\u1ee5m.<\/p>\n<p>Ngo\u00e0i c\u00f4ng c\u1ee5 x\u1eed l\u00fd d\u1eef li\u1ec7u Spark, c\u00f2n c\u00f3 c\u00e1c th\u01b0 vi\u1ec7n d\u00e0nh cho SQL v\u00e0 DataFrames, h\u1ecdc m\u00e1y, GraphX, t\u00ednh to\u00e1n bi\u1ec3u \u0111\u1ed3 v\u00e0 x\u1eed l\u00fd lu\u1ed3ng.\u00a0C\u00e1c th\u01b0 vi\u1ec7n n\u00e0y c\u00f3 th\u1ec3 \u0111\u01b0\u1ee3c s\u1eed d\u1ee5ng c\u00f9ng nhau tr\u00ean c\u00e1c t\u1eadp d\u1eef li\u1ec7u l\u1edbn t\u1eeb nhi\u1ec1u ngu\u1ed3n d\u1eef li\u1ec7u kh\u00e1c nhau, ch\u1eb3ng h\u1ea1n nh\u01b0 HDFS, Alluxio, Apache Cassandra, Apache HBase ho\u1eb7c Apache Hive.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/tgmc-blog-64fe9d24abc4b.png\" alt=\"\" \/><\/p>\n<p><em>C\u00e1c th\u00e0nh ph\u1ea7n Apache Spark<\/em><\/p>\n<h2>T\u1ea1i sao c\u1ea7n \u0111\u1ebfn Apache Spark?<\/h2>\n<p>Apache Spark ti\u1ebfp t\u1ee5c n\u1ed7 l\u1ef1c ph\u00e2n t\u00edch d\u1eef li\u1ec7u l\u1edbn m\u00e0 Apache Hadoop \u0111\u00e3 b\u1eaft \u0111\u1ea7u t\u1eeb h\u01a1n 15 n\u0103m tr\u01b0\u1edbc v\u00e0 \u0111\u00e3 tr\u1edf th\u00e0nh framework h\u00e0ng \u0111\u1ea7u \u0111\u1ec3 x\u1eed l\u00fd d\u1eef li\u1ec7u ph\u00e2n t\u00e1n \u1edf quy m\u00f4 l\u1edbn.<\/p>\n<p>Vi\u1ec7c s\u1eed d\u1ee5ng Hadoop ng\u00e0y c\u00e0ng t\u0103ng c\u00f9ng v\u1edbi s\u1ef1 ph\u1ed5 bi\u1ebfn c\u1ee7a ph\u00e2n t\u00edch d\u1eef li\u1ec7u l\u1edbn v\u00e0o \u0111\u1ea7u nh\u1eefng n\u0103m 2010, c\u00e1c h\u1ea1n ch\u1ebf v\u1ec1 hi\u1ec7u su\u1ea5t c\u1ee7a Hadoop MapReduce \u0111\u00e3 tr\u1edf th\u00e0nh m\u1ed9t \u0111i\u1ec3m y\u1ebfu, b\u1ecb t\u1eafc ngh\u1ebdn b\u1edfi m\u00f4 h\u00ecnh xu\u1ea5t k\u1ebft qu\u1ea3 checkpoint v\u00e0o \u1ed5 l\u01b0u tr\u1eef.\u00a0\u0110\u1ed3ng th\u1eddi vi\u1ec7c \u00e1p d\u1ee5ng Hadoop \u0111\u00e3 b\u1ecb c\u1ea3n tr\u1edf b\u1edfi m\u00f4 h\u00ecnh l\u1eadp tr\u00ecnh c\u1ea5p th\u1ea5p c\u1ee7a MapReduce.<\/p>\n<p>Apache Spark b\u1eaft \u0111\u1ea7u nh\u01b0 m\u1ed9t\u00a0<a href=\"https:\/\/spark.apache.org\/research.html\" target=\"_blank\" rel=\"noopener noreferrer\">d\u1ef1 \u00e1n nghi\u00ean c\u1ee9u<\/a>\u00a0t\u1ea1i UC Berkeley trong AMPLab, v\u1edbi m\u1ee5c ti\u00eau duy tr\u00ec c\u00e1c l\u1ee3i \u00edch c\u1ee7a framework x\u1eed l\u00fd c\u00f3 kh\u1ea3 n\u0103ng m\u1edf r\u1ed9ng, ph\u00e2n t\u00e1n, c\u00f3 kh\u1ea3 n\u0103ng ch\u1ecbu l\u1ed7i c\u1ee7a MapReduce, \u0111\u1ed3ng th\u1eddi hi\u1ec7u qu\u1ea3 h\u01a1n v\u00e0 d\u1ec5 s\u1eed d\u1ee5ng h\u01a1n.\u00a0Spark hi\u1ec7u qu\u1ea3 h\u01a1n MapReduce \u0111\u1ed1i v\u1edbi c\u00e1c data pipelines v\u00e0 thu\u1eadt to\u00e1n l\u1eb7p v\u00ec n\u00f3 s\u1eed d\u1ee5ng c\u00e1c t\u00e1c v\u1ee5 nh\u1eb9 \u0111a lu\u1ed3ng. L\u01b0u tr\u1eef d\u1eef li\u1ec7u trong b\u1ed9 nh\u1edb qua c\u00e1c l\u1ea7n l\u1eb7p, lo\u1ea1i b\u1ecf nhu c\u1ea7u ghi v\u00e0o \u0111\u0129a gi\u1eefa c\u00e1c giai \u0111o\u1ea1n.\u00a0Spark s\u1eed d\u1ee5ng DataFrame ph\u00e2n t\u00e1n c\u00f3 kh\u1ea3 n\u0103ng ch\u1ecbu l\u1ed7i \u0111\u1ec3 n\u00e2ng cao hi\u1ec7u su\u1ea5t song song v\u00e0 gi\u00fap d\u1ec5 s\u1eed d\u1ee5ng v\u1edbi SQL.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/tgmc-blog-64fe9d25656f5.png\" alt=\"\" \/><\/p>\n<p>Spark l\u00e0 <a href=\"https:\/\/www.apache.org\/\">n\u1ec1n t\u1ea3ng ph\u1ea7n m\u1ec1m<\/a>\u00a0\u1edf c\u1ea5p cao nh\u1ea5t v\u00e0o n\u0103m 2014 \u0111\u1ebfn nay, h\u00e0ng tr\u0103m ngh\u00ecn k\u1ef9 s\u01b0 d\u1eef li\u1ec7u v\u00e0 nh\u00e0 khoa h\u1ecdc \u0111ang l\u00e0m vi\u1ec7c v\u1edbi Spark tr\u00ean h\u01a1n 16.000 doanh nghi\u1ec7p v\u00e0 t\u1ed5 ch\u1ee9c.\u00a0M\u1ed9t l\u00fd do khi\u1ebfn Spark chi\u1ebfm \u01b0u th\u1ebf h\u01a1n Hadoop l\u00e0 v\u00ec qu\u00e1 tr\u00ecnh x\u1eed l\u00fd d\u1eef li\u1ec7u trong b\u1ed9 nh\u1edb c\u00f3 th\u1ec3 ho\u00e0n th\u00e0nh m\u1ed9t s\u1ed1 t\u00e1c v\u1ee5 nhanh h\u01a1n t\u1edbi 100 l\u1ea7n so v\u1edbi MapReduce.\u00a0Nh\u1eefng kh\u1ea3 n\u0103ng n\u00e0y \u0111\u01b0\u1ee3c t\u1ea1o ra trong m\u1ed9t c\u1ed9ng \u0111\u1ed3ng m\u1edf b\u1edfi h\u01a1n 1.000 ng\u01b0\u1eddi \u0111\u00f3ng g\u00f3p tr\u00ean h\u01a1n 250 c\u00f4ng ty.\u00a0Nh\u1eefng ng\u01b0\u1eddi s\u00e1ng l\u1eadp Databricks \u0111\u00e3 b\u1eaft \u0111\u1ea7u n\u1ed7 l\u1ef1c v\u00e0 ch\u1ec9 ri\u00eang n\u1ec1n t\u1ea3ng c\u1ee7a h\u1ecd \u0111\u00e3 t\u1ea1o ra h\u01a1n 1 tri\u1ec7u m\u00e1y \u1ea3o m\u1ed7i ng\u00e0y \u0111\u1ec3 ph\u00e2n t\u00edch d\u1eef li\u1ec7u.<\/p>\n<h2>T\u1ea1i sao hi\u1ec7u su\u1ea5t c\u1ee7a Spark t\u1ed1t h\u01a1n v\u1edbi GPU?<\/h2>\n<p>V\u1edbi m\u1ed7i b\u1ea3n ph\u00e1t h\u00e0nh Spark, c\u00e1c c\u1ea3i ti\u1ebfn \u0111\u00e3 \u0111\u01b0\u1ee3c tri\u1ec3n khai \u0111\u1ec3 gi\u00fap l\u1eadp tr\u00ecnh d\u1ec5 d\u00e0ng h\u01a1n v\u00e0 th\u1ef1c thi nhanh h\u01a1n.\u00a0Apache Spark 3.0 ti\u1ebfp t\u1ee5c xu h\u01b0\u1edbng n\u00e0y v\u1edbi nh\u1eefng c\u1ea3i ti\u1ebfn nh\u1eb1m c\u1ea3i thi\u1ec7n hi\u1ec7u su\u1ea5t Spark SQL v\u00e0 kh\u1ea3 n\u0103ng t\u0103ng t\u1ed1c b\u1edfi GPU c\u1ee7a NVIDIA.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/tgmc-blog-64fe9d26b2e8e.png\" alt=\"\" \/><\/p>\n<p>B\u1ed9 x\u1eed l\u00fd \u0111\u1ed3 h\u1ecda (GPU) ph\u1ed5 bi\u1ebfn nh\u1edd m\u1ee9c gi\u00e1 th\u1ea5p v\u00e0 \u0111ang gi\u1ea3i quy\u1ebft c\u00e1c v\u1ea5n \u0111\u1ec1 hi\u1ec7u su\u1ea5t \u0111i\u1ec7n to\u00e1n ng\u00e0y nay b\u1eb1ng c\u00e1ch t\u0103ng t\u1ed1c c\u00e1c m\u00e1y ch\u1ee7 \u0111a nh\u00e2n \u0111\u1ec3 x\u1eed l\u00fd song song.\u00a0CPU g\u1ed3m m\u1ed9t s\u1ed1 core \u0111\u01b0\u1ee3c t\u1ed1i \u01b0u h\u00f3a \u0111\u1ec3 x\u1eed l\u00fd n\u1ed1i ti\u1ebfp.\u00a0Nh\u01b0ng GPU c\u00f3 ki\u1ebfn \u200b\u200btr\u00fac song song l\u1edbn bao g\u1ed3m h\u00e0ng ngh\u00ecn core nh\u1ecf h\u01a1n, hi\u1ec7u qu\u1ea3 h\u01a1n \u0111\u01b0\u1ee3c thi\u1ebft k\u1ebf \u0111\u1ec3 x\u1eed l\u00fd nhi\u1ec1u t\u00e1c v\u1ee5 c\u00f9ng m\u1ed9t l\u00fac.\u00a0Ch\u00fang c\u00f3 kh\u1ea3 n\u0103ng x\u1eed l\u00fd d\u1eef li\u1ec7u nhanh h\u01a1n nhi\u1ec1u so v\u1edbi c\u1ea5u h\u00ecnh ch\u1ec9 c\u00f3 CPU.\u00a0GPU \u0111\u00e3 g\u00f3p ph\u1ea7n th\u00fac \u0111\u1ea9y s\u1ef1 ph\u00e1t tri\u1ec3n c\u1ee7a \u0111\u00e0o t\u1ea1o m\u00f4 h\u00ecnh h\u1ecdc s\u00e2u (DL) v\u00e0 h\u1ecdc m\u00e1y(ML) trong v\u00e0i n\u0103m qua.\u00a0Tuy nhi\u00ean,\u00a0<a href=\"https:\/\/towardsdatascience.com\/workflow-of-a-machine-learning-project-ec1dba419b94\" target=\"_blank\" rel=\"noopener noreferrer\">80% th\u1eddi gian c\u1ee7a nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u v\u1eabn d\u00e0nh cho vi\u1ec7c x\u1eed l\u00fd tr\u01b0\u1edbc (preprocessing) d\u1eef li\u1ec7u<\/a>\u00a0.<\/p>\n<p>Trong khi Spark t\u00ednh to\u00e1n tr\u00ean c\u00e1c node d\u01b0\u1edbi d\u1ea1ng ph\u00e2n v\u00f9ng, trong m\u1ed9t ph\u00e2n v\u00f9ng, vi\u1ec7c t\u00ednh to\u00e1n tr\u01b0\u1edbc \u0111\u00e2y \u0111\u01b0\u1ee3c th\u1ef1c hi\u1ec7n tr\u00ean c\u00e1c core CPU.\u00a0Spark \u0111\u00e3 gi\u1ea3m thi\u1ec3u c\u00e1c v\u1ea5n \u0111\u1ec1 I\/O g\u1eb7p ph\u1ea3i trong Hadoop b\u1eb1ng c\u00e1ch th\u00eam t\u00ednh n\u0103ng x\u1eed l\u00fd d\u1eef li\u1ec7u trong b\u1ed9 nh\u1edb, nh\u01b0ng gi\u1edd \u0111\u00e2y, vi\u1ec7c ngh\u1ebdn c\u1ed5 chai \u0111\u00e3 chuy\u1ec3n t\u1eeb th\u00e0nh ph\u1ea7n I\/O sang th\u00e0nh ph\u1ea7n x\u1eed l\u00fd cho s\u1ed1 l\u01b0\u1ee3ng \u1ee9ng d\u1ee5ng ng\u00e0y c\u00e0ng t\u0103ng.\u00a0V\u1ea5n \u0111\u1ec1 v\u1ec1 hi\u1ec7u su\u1ea5t n\u00e0y c\u00f3 th\u1ec3 \u0111\u01b0\u1ee3c kh\u1eafc ph\u1ee5c nh\u1edd s\u1ef1 k\u1ebft h\u1ee3p v\u1edbi GPU.<\/p>\n<p>\u0110\u1ec3 \u0111\u00e1p \u1ee9ng v\u00e0 v\u01b0\u1ee3t qua c\u00e1c y\u00eau c\u1ea7u v\u1ec1 x\u1eed l\u00fd d\u1eef li\u1ec7u,<a href=\"https:\/\/cloud.google.com\/blog\/topics\/partners\/google-cloud-supports-ampere-architecture-and-a100-tensor-core-gpu\" target=\"_blank\" rel=\"noopener noreferrer\">NVIDIA<\/a>\u00a0\u0111\u00e3 c\u1ed9ng t\u00e1c v\u1edbi c\u1ed9ng \u0111\u1ed3ng Apache Spark \u0111\u1ec3\u00a0<a href=\"https:\/\/www.nvidia.com\/en-us\/deep-learning-ai\/solutions\/data-science\/apache-spark-3\/\">\u0111\u01b0a GPU v\u00e0o qu\u00e1 tr\u00ecnh x\u1eed l\u00fd g\u1ed1c c\u1ee7a Spark<\/a>\u00a0th\u00f4ng qua vi\u1ec7c ph\u00e1t h\u00e0nh Spark 3.0 v\u00e0 c\u00f4ng c\u1ee5 t\u0103ng t\u1ed1c NVIDIA RAPIDS m\u00e3 ngu\u1ed3n m\u1edf cho Spark.\u00a0L\u1ee3i \u00edch c\u1ee7a vi\u1ec7c c\u1ea3i thi\u1ec7n hi\u1ec7u su\u1ea5t Spark v\u1edbi GPU l\u00e0 r\u1ea5t l\u1edbn:<\/p>\n<ul>\n<li>Qu\u00e1 tr\u00ecnh x\u1eed l\u00fd d\u1eef li\u1ec7u, truy v\u1ea5n v\u00e0 \u0111\u00e0o t\u1ea1o m\u00f4 h\u00ecnh \u0111\u01b0\u1ee3c ho\u00e0n th\u00e0nh nhanh h\u01a1n, gi\u1ea3m th\u1eddi gian \u0111\u01b0a ra k\u1ebft qu\u1ea3.<\/li>\n<li>C\u01a1 s\u1edf h\u1ea1 t\u1ea7ng \u0111\u01b0\u1ee3c t\u0103ng t\u1ed1c b\u1edfi GPU c\u00f3 th\u1ec3 \u0111\u01b0\u1ee3c s\u1eed d\u1ee5ng cho c\u1ea3 framework Spark v\u00e0 ML\/DL, lo\u1ea1i b\u1ecf nhu c\u1ea7u v\u1ec1 c\u00e1c c\u1ee5m ri\u00eang bi\u1ec7t v\u00e0 cho ph\u00e9p to\u00e0n b\u1ed9 quy tr\u00ecnh truy c\u1eadp v\u00e0o kh\u1ea3 n\u0103ng t\u0103ng t\u1ed1c GPU.<\/li>\n<li>C\u1ea7n \u00edt m\u00e1y ch\u1ee7 h\u01a1n, gi\u1ea3m chi ph\u00ed c\u01a1 s\u1edf h\u1ea1 t\u1ea7ng.<\/li>\n<\/ul>\n<h2>C\u00f4ng c\u1ee5 t\u0103ng t\u1ed1c RAPIDS cho Apache Spark<\/h2>\n<p><a href=\"https:\/\/rapids.ai\/\" target=\"_blank\" rel=\"noopener noreferrer\">RAPIDS<\/a>\u00a0l\u00e0 m\u1ed9t b\u1ed9 th\u01b0 vi\u1ec7n ph\u1ea7n m\u1ec1m ngu\u1ed3n m\u1edf v\u00e0 API \u0111\u1ec3 th\u1ef1c thi quy tr\u00ecnh ph\u00e2n t\u00edch v\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u t\u1eeb \u0111\u1ea7u \u0111\u1ebfn cu\u1ed1i ho\u00e0n to\u00e0n tr\u00ean GPU, cho ph\u00e9p t\u0103ng t\u1ed1c \u0111\u00e1ng k\u1ec3, \u0111\u1eb7c bi\u1ec7t l\u00e0 tr\u00ean c\u00e1c t\u1eadp d\u1eef li\u1ec7u l\u1edbn.\u00a0\u0110\u01b0\u1ee3c x\u00e2y d\u1ef1ng d\u1ef1a tr\u00ean\u00a0\u00a0<a href=\"https:\/\/developer.nvidia.com\/cuda-zone\">NVIDIA CUDA<\/a>\u00a0v\u00e0\u00a0<a href=\"https:\/\/www.openucx.org\/\">UCX<\/a>,\u00a0<a href=\"https:\/\/www.nvidia.com\/en-us\/deep-learning-ai\/solutions\/data-science\/apache-spark-3\/\">C\u00f4ng c\u1ee5 t\u0103ng t\u1ed1c RAPIDS d\u00e0nh cho Apache Spark<\/a>\u00a0cho ph\u00e9p c\u00e1c ho\u1ea1t \u0111\u1ed9ng SQL\/DataFrame \u0111\u01b0\u1ee3c t\u0103ng t\u1ed1c b\u1edfi GPU v\u00e0 c\u00e1c bi\u1ebfn \u0111\u1ed9ng tr\u00ean Spark m\u00e0 kh\u00f4ng c\u1ea7n thay \u0111\u1ed5i m\u00e3 l\u1eadp tr\u00ecnh.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/tgmc-blog-64fe9d27208a2.png\" alt=\"\" \/><\/p>\n<h2>SQL\/DataFrame \u0111\u01b0\u1ee3c c\u1ea3i thi\u1ec7n<\/h2>\n<p>Spark 3.0 h\u1ed7 tr\u1ee3 c\u00e1c tr\u00ecnh t\u1ed1i \u01b0u h\u00f3a SQL \u0111\u1ec3 x\u1eed l\u00fd d\u1eef li\u1ec7u b\u1eb1ng c\u00e1ch s\u1eed d\u1ee5ng c\u00e1c t\u1eadp columnar thay v\u00ec rows.\u00a0D\u1eef li\u1ec7u\u00a0columnar th\u00e2n thi\u1ec7n v\u1edbi GPU v\u00e0 t\u00ednh n\u0103ng n\u00e0y l\u00e0 th\u1ee9 m\u00e0 c\u00f4ng c\u1ee5 t\u0103ng t\u1ed1c RAPIDS t\u00edch h\u1ee3p \u0111\u1ec3 t\u0103ng t\u1ed1c c\u00e1c to\u00e1n t\u1eed SQL v\u00e0 DataFrame.\u00a0V\u1edbi tr\u00ecnh t\u0103ng t\u1ed1c RAPIDS, tr\u00ecnh t\u1ed1i \u01b0u h\u00f3a truy v\u1ea5n Catalyst \u0111\u00e3 \u0111\u01b0\u1ee3c s\u1eeda \u0111\u1ed5i \u0111\u1ec3 x\u00e1c \u0111\u1ecbnh c\u00e1c to\u00e1n t\u1eed trong g\u00f3i truy v\u1ea5n c\u00f3 th\u1ec3 \u0111\u01b0\u1ee3c t\u0103ng t\u1ed1c b\u1eb1ng API RAPIDS, ch\u1ee7 y\u1ebfu l\u00e0 \u00e1nh x\u1ea1 m\u1ed9t-m\u1ed9t v\u00e0 l\u00ean l\u1ecbch cho c\u00e1c to\u00e1n t\u1eed \u0111\u00f3 tr\u00ean GPU trong c\u1ee5m Spark khi th\u1ef1c thi k\u1ebf ho\u1ea1ch truy v\u1ea5n.<\/p>\n<h2>T\u0103ng t\u1ed1c c\u00e1c ho\u1ea1t \u0111\u1ed9ng Shuffle<\/h2>\n<p>Spark th\u1ef1c hi\u1ec7n vi\u1ec7c s\u1eafp x\u1ebfp, ph\u00e2n nh\u00f3m ho\u1eb7c n\u1ed1i d\u1eef li\u1ec7u theo gi\u00e1 tr\u1ecb ph\u1ea3i di chuy\u1ec3n d\u1eef li\u1ec7u gi\u1eefa c\u00e1c ph\u00e2n v\u00f9ng khi t\u1ea1o DataFrame m\u1edbi t\u1eeb DataFrame hi\u1ec7n c\u00f3 gi\u1eefa c\u00e1c giai \u0111o\u1ea1n.\u00a0Qu\u00e1 tr\u00ecnh n\u00e0y \u0111\u01b0\u1ee3c g\u1ecdi l\u00e0 Shuffle v\u00e0 bao g\u1ed3m I\/O v\u1edbi \u1ed5 l\u01b0u tr\u1eef, tu\u1ea7n t\u1ef1 h\u00f3a d\u1eef li\u1ec7u v\u00e0 I\/O m\u1ea1ng.\u00a0Vi\u1ec7c tri\u1ec3n khai x\u00e1o RAPIDS Accelerator shuffle m\u1edbi s\u1eed d\u1ee5ng\u00a0<a href=\"https:\/\/www.openucx.org\/\">UCX<\/a>\u00a0\u0111\u1ec3 t\u1ed1i \u01b0u h\u00f3a vi\u1ec7c truy\u1ec1n d\u1eef li\u1ec7u GPU, duy tr\u00ec c\u00e0ng nhi\u1ec1u d\u1eef li\u1ec7u tr\u00ean GPU c\u00e0ng t\u1ed1t. T\u00ecm ra con \u0111\u01b0\u1eddng nhanh nh\u1ea5t \u0111\u1ec3 di chuy\u1ec3n d\u1eef li\u1ec7u gi\u1eefa c\u00e1c node b\u1eb1ng c\u00e1ch s\u1eed d\u1ee5ng t\u1ed1t nh\u1ea5t c\u00e1c t\u00e0i nguy\u00ean ph\u1ea7n c\u1ee9ng s\u1eb5n c\u00f3, bao g\u1ed3m c\u1ea3 vi\u1ec7c b\u1ecf qua CPU \u0111\u1ec3 th\u1ef1c hi\u1ec7n chuy\u1ec3n GPU sang b\u1ed9 nh\u1edb GPU gi\u1eefa c\u00e1c node.<\/p>\n<h2>Accelerator-aware Scheduling (L\u1eadp l\u1ecbch c\u00f3 s\u1ef1 h\u1ed7 tr\u1ee3 c\u1ee7a b\u1ed9 t\u0103ng t\u1ed1c)<\/h2>\n<p>L\u00e0 m\u1ed9t ph\u1ea7n s\u00e1ng ki\u1ebfn \u200b\u200bquan tr\u1ecdng c\u1ee7a Spark nh\u1eb1m h\u1ee3p nh\u1ea5t t\u1ed1t vi\u1ec7c h\u1ecdc s\u00e2u v\u00e0 x\u1eed l\u00fd d\u1eef li\u1ec7u tr\u00ean Spark, GPU hi\u1ec7n l\u00e0 t\u00e0i nguy\u00ean c\u00f3 th\u1ec3 l\u1eadp l\u1ecbch tr\u00ecnh trong Apache Spark 3.0.\u00a0\u0110i\u1ec1u n\u00e0y cho ph\u00e9p Spark l\u00ean l\u1ecbch cho b\u1ed9 th\u1ef1c thi v\u1edbi s\u1ed1 l\u01b0\u1ee3ng GPU \u0111\u01b0\u1ee3c ch\u1ec9 \u0111\u1ecbnh v\u00e0 cho ph\u00e9p ng\u01b0\u1eddi d\u00f9ng ch\u1ec9 \u0111\u1ecbnh s\u1ed1 l\u01b0\u1ee3ng GPU m\u00e0 m\u1ed7i t\u00e1c v\u1ee5 y\u00eau c\u1ea7u.\u00a0Spark chuy\u1ec3n c\u00e1c y\u00eau c\u1ea7u t\u00e0i nguy\u00ean n\u00e0y \u0111\u1ebfn tr\u00ecnh qu\u1ea3n l\u00fd c\u1ee5m\u00a0c\u01a1 b\u1ea3n, Kubernetes, YARN ho\u1eb7c Standalone.\u00a0Ng\u01b0\u1eddi d\u00f9ng c\u0169ng c\u00f3 th\u1ec3 \u0111\u1ecbnh c\u1ea5u h\u00ecnh t\u1eadp l\u1ec7nh kh\u00e1m ph\u00e1 c\u00f3 th\u1ec3 ph\u00e1t hi\u1ec7n GPU n\u00e0o \u0111\u01b0\u1ee3c ng\u01b0\u1eddi qu\u1ea3n l\u00fd c\u1ee5m ch\u1ec9 \u0111\u1ecbnh.\u00a0\u0110i\u1ec1u n\u00e0y gi\u00fap \u0111\u01a1n gi\u1ea3n h\u00f3a \u0111\u00e1ng k\u1ec3 vi\u1ec7c ch\u1ea1y c\u00e1c \u1ee9ng d\u1ee5ng ML c\u1ea7n GPU, v\u00ec ng\u01b0\u1eddi d\u00f9ng tr\u01b0\u1edbc \u0111\u00e2y ph\u1ea3i gi\u1ea3i quy\u1ebft v\u1ea5n \u0111\u1ec1 thi\u1ebfu l\u1eadp l\u1ecbch GPU trong c\u00e1c \u1ee9ng d\u1ee5ng Spark.<\/p>\n<h2>XGBoost \u0111\u01b0\u1ee3c c\u1ea3i thi\u1ec7n<\/h2>\n<p><a href=\"https:\/\/rapids.ai\/xgboost.html\" target=\"_blank\" rel=\"noopener noreferrer\">XGBoos<\/a>\u00a0c\u00f3 kh\u1ea3 n\u0103ng m\u1edf r\u1ed9ng, ph\u00e2n b\u1ed5 <a href=\"https:\/\/en.wikipedia.org\/wiki\/Gradient_boosting\" target=\"_blank\" rel=\"noopener noreferrer\">gradient-boosted<\/a>\u00a0v\u1edbi th\u01b0 vi\u1ec7n ML. L\u00e0 th\u01b0 vi\u1ec7n ML h\u00e0ng \u0111\u1ea7u cho vi\u1ec7c ph\u00e2n t\u00edch h\u1ed3i quy, ph\u00e2n lo\u1ea1i v\u00e0 x\u1ebfp h\u1ea1ng.\u00a0Nh\u00f3m RAPIDS h\u1ee3p t\u00e1c ch\u1eb7t ch\u1ebd v\u1edbi t\u1ed5 ch\u1ee9c XGBoost Common Machine Learning (DMLC) v\u00e0 XGBoost hi\u1ec7n bao g\u1ed3m kh\u1ea3 n\u0103ng t\u0103ng t\u1ed1c GPU m\u1ed9t c\u00e1ch li\u1ec1n m\u1ea1ch, c\u00f3 th\u1ec3 c\u00e0i \u0111\u1eb7t s\u1eb5n.\u00a0Spark 3.0 XGBoost hi\u1ec7n c\u0169ng \u0111\u01b0\u1ee3c t\u00edch h\u1ee3p v\u1edbi Rapids Accelerator c\u1ea3i thi\u1ec7n hi\u1ec7u su\u1ea5t, \u0111\u1ed9 ch\u00ednh x\u00e1c v\u00e0 chi ph\u00ed v\u1edbi kh\u1ea3 n\u0103ng t\u0103ng t\u1ed1c GPU c\u1ee7a c\u00e1c ho\u1ea1t \u0111\u1ed9ng Spark SQL\/DataFrame, t\u0103ng t\u1ed1c GPU trong th\u1eddi gian \u0111\u00e0o t\u1ea1o XGBoost v\u00e0 s\u1eed d\u1ee5ng b\u1ed9 nh\u1edb GPU hi\u1ec7u qu\u1ea3 v\u1edbi c\u00e1c t\u00ednh n\u0103ng \u0111\u01b0\u1ee3c l\u01b0u tr\u1eef t\u1ed1i \u01b0u trong b\u1ed9 nh\u1edb.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/tgmc-blog-64fe9d27d1e1f.png\" alt=\"\" \/><\/p>\n<p>V\u1edbi Spark 3.0, gi\u1edd \u0111\u00e2y b\u1ea1n c\u00f3 th\u1ec3 c\u00f3 m\u1ed9t quy tr\u00ecnh duy nh\u1ea5t, t\u1eeb nh\u1eadp d\u1eef li\u1ec7u \u0111\u1ebfn chu\u1ea9n b\u1ecb d\u1eef li\u1ec7u cho \u0111\u1ebfn \u0111\u00e0o t\u1ea1o m\u00f4 h\u00ecnh tr\u00ean c\u1ee5m h\u1ed7 tr\u1ee3 v\u1edbi GPU<\/p>\n<p>&nbsp;<\/p>\n<h2>Khoa h\u1ecdc d\u1eef li\u1ec7u to\u00e0n di\u1ec7n, \u0111\u01b0\u1ee3c t\u0103ng t\u1ed1c v\u1edbi NVIDIA GPU<\/h2>\n<p>RAPIDS tr\u1eebu t\u01b0\u1ee3ng h\u00f3a s\u1ef1 ph\u1ee9c t\u1ea1p c\u1ee7a khoa h\u1ecdc d\u1eef li\u1ec7u \u0111\u01b0\u1ee3c c\u1ea3i thi\u1ec7n b\u1eb1ng c\u00e1ch x\u00e2y d\u1ef1ng v\u00e0 t\u00edch h\u1ee3p v\u1edbi c\u00e1c h\u1ec7 sinh th\u00e1i ph\u00e2n t\u00edch ph\u1ed5 bi\u1ebfn nh\u01b0 PyData v\u00e0 Apache Spark, cho ph\u00e9p ng\u01b0\u1eddi d\u00f9ng th\u1ea5y \u0111\u01b0\u1ee3c l\u1ee3i \u00edch ngay l\u1eadp t\u1ee9c.\u00a0So v\u1edbi c\u00e1c tri\u1ec3n khai d\u1ef1a tr\u00ean CPU t\u01b0\u01a1ng t\u1ef1,\u00a0<a href=\"https:\/\/www.nvidia.com\/en-us\/deep-learning-ai\/software\/rapids\/\">RAPIDS<\/a>\u00a0mang l\u1ea1i c\u1ea3i ti\u1ebfn hi\u1ec7u su\u1ea5t g\u1ea5p 50 l\u1ea7n cho c\u00e1c quy tr\u00ecnh ph\u00e2n t\u00edch d\u1eef li\u1ec7u c\u1ed5 \u0111i\u1ec3n v\u00e0 h\u1ecdc m\u00e1y (ML) tr\u00ean quy m\u00f4 l\u1edbn, gi\u00fap gi\u1ea3m \u0111\u00e1ng k\u1ec3 t\u1ed5ng chi ph\u00ed s\u1edf h\u1eefu (TCO) cho c\u00e1c ho\u1ea1t \u0111\u1ed9ng khoa h\u1ecdc d\u1eef li\u1ec7u l\u1edbn.<\/p>\n<h2>C\u00e1c v\u00ed d\u1ee5 th\u1ef1c t\u1ebf v\u1ec1 vi\u1ec7c t\u0103ng t\u1ed1c qu\u00e1 tr\u00ecnh h\u1ecdc m\u00e1y t\u1eeb \u0111\u1ea7u \u0111\u1ebfn cu\u1ed1i v\u1edbi Spark<\/h2>\n<h2>Adobe<\/h2>\n<p>D\u1ef1a tr\u00ean quan h\u1ec7 \u0111\u1ed1i t\u00e1c chi\u1ebfn l\u01b0\u1ee3c AI v\u1edbi NVIDIA, Adobe l\u00e0 m\u1ed9t trong nh\u1eefng c\u00f4ng ty \u0111\u1ea7u ti\u00ean h\u1ee3p t\u00e1c v\u1edbi b\u1ea3n ph\u00e1t h\u00e0nh c\u1ee7a Spark 3.0 ch\u1ea1y tr\u00ean Databricks.\u00a0T\u1ea1i\u00a0<a href=\"https:\/\/developer.nvidia.com\/gtc\/2020\/video\/s22661\">h\u1ed9i ngh\u1ecb NVIDIA GTC, Adobe Intelligence Services<\/a>\u00a0\u0111\u00e3 cung c\u1ea5p k\u1ebft qu\u1ea3 \u0111\u00e1nh gi\u00e1 v\u1ec1 gi\u1ea3i ph\u00e1p email th\u00f4ng minh Spark 3.0 v\u00e0 XGBoost d\u1ef1a tr\u00ean GPU \u0111\u1ec3 t\u1ed1i \u01b0u h\u00f3a vi\u1ec7c g\u1eedi th\u00f4ng \u0111i\u1ec7p ti\u1ebfp th\u1ecb.\u00a0Trong c\u00e1c th\u1eed nghi\u1ec7m ban \u0111\u1ea7u, Adobe \u0111\u00e3 \u0111\u1ea1t \u0111\u01b0\u1ee3c m\u1ee9c c\u1ea3i thi\u1ec7n hi\u1ec7u su\u1ea5t g\u1ea5p 7 l\u1ea7n v\u00e0 ti\u1ebft ki\u1ec7m 90% chi ph\u00ed.\u00a0Hi\u1ec7u su\u1ea5t \u0111\u1ea1t \u0111\u01b0\u1ee3c trong Spark 3.0 n\u00e2ng cao \u0111\u1ed9 ch\u00ednh x\u00e1c c\u1ee7a m\u00f4 h\u00ecnh b\u1eb1ng c\u00e1ch cho ph\u00e9p c\u00e1c nh\u00e0 khoa h\u1ecdc \u0111\u00e0o t\u1ea1o c\u00e1c m\u00f4 h\u00ecnh v\u1edbi b\u1ed9 d\u1eef li\u1ec7u l\u1edbn h\u01a1n v\u00e0 \u0111\u00e0o t\u1ea1o l\u1ea1i m\u00f4 h\u00ecnh th\u01b0\u1eddng xuy\u00ean h\u01a1n.\u00a0\u0110i\u1ec1u n\u00e0y gi\u00fap x\u1eed l\u00fd h\u00e0ng terabyte d\u1eef li\u1ec7u m\u1edbi m\u1ed7i ng\u00e0y, \u0111i\u1ec1u n\u00e0y r\u1ea5t quan tr\u1ecdng \u0111\u1ed1i v\u1edbi c\u00e1c nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u h\u1ed7 tr\u1ee3 h\u1ec7 th\u1ed1ng \u0111\u1ec1 xu\u1ea5t tr\u1ef1c tuy\u1ebfn ho\u1eb7c ph\u00e2n t\u00edch d\u1eef li\u1ec7u nghi\u00ean c\u1ee9u m\u1edbi.\u00a0Ngo\u00e0i ra, x\u1eed l\u00fd nhanh h\u01a1n c\u00f3 ngh\u0129a l\u00e0 c\u1ea7n \u00edt t\u00e0i nguy\u00ean ph\u1ea7n c\u1ee9ng h\u01a1n \u0111\u1ec3 mang l\u1ea1i k\u1ebft qu\u1ea3, gi\u00fap ti\u1ebft ki\u1ec7m chi ph\u00ed \u0111\u00e1ng k\u1ec3.\u00a0Khi \u0111\u01b0\u1ee3c h\u1ecfi v\u1ec1 nh\u1eefng ti\u1ebfn b\u1ed9 n\u00e0y, William Yan, gi\u00e1m \u0111\u1ed1c c\u1ea5p cao v\u1ec1 Machine Learning t\u1ea1i Adobe cho bi\u1ebft:\u00a0\u201cCh\u00fang t\u00f4i nh\u1eadn th\u1ea5y hi\u1ec7u su\u1ea5t nhanh h\u01a1n \u0111\u00e1ng k\u1ec3 v\u1edbi Spark 3.0 \u0111\u01b0\u1ee3c t\u0103ng t\u1ed1c b\u1edfi NVIDIA so v\u1edbi ch\u1ea1y Spark v\u1edbi CPU.\u00a0V\u1edbi nh\u1eefng m\u1ee9c t\u0103ng hi\u1ec7u su\u1ea5t GPU, nh\u1eefng kh\u1ea3 n\u0103ng ho\u00e0n to\u00e0n m\u1edbi s\u1ebd m\u1edf ra \u0111\u1ec3 n\u00e2ng cao c\u00e1c t\u00ednh n\u0103ng do AI \u0111i\u1ec1u khi\u1ec3n trong b\u1ed9 \u1ee9ng d\u1ee5ng Adobe Experience Cloud.\u201d<\/p>\n<h2>Verizon media<\/h2>\n<p><a href=\"https:\/\/resources.nvidia.com\/gtcd-2020\/GTC2020-s22446?lx=3X9y6T\">Verizon Media x\u00e2y d\u1ef1ng<\/a>\u00a0quy tr\u00ecnh Spark ML ph\u00e2n t\u00e1n \u0111\u1ec3 \u0111\u00e0o t\u1ea1o m\u00f4 h\u00ecnh XGBoost v\u00e0 \u0111i\u1ec1u ch\u1ec9nh si\u00eau tham s\u1ed1 tr\u00ean c\u1ee5m d\u1ef1a v\u1edbi GPU.\u00a0Verizon Media \u0111\u00e3 \u0111\u1ea1t \u0111\u01b0\u1ee3c m\u1ee9c c\u1ea3i thi\u1ec7n hi\u1ec7u su\u1ea5t g\u1ea5p 3 l\u1ea7n so v\u1edbi gi\u1ea3i ph\u00e1p XGBoost ch\u1ec9 c\u00f3 CPU, n\u00e2ng cao kh\u1ea3 n\u0103ng th\u1ef1c hi\u1ec7n t\u00ecm ki\u1ebfm si\u00eau tham s\u1ed1 nh\u1eb1m t\u00ecm ra si\u00eau tham s\u1ed1 t\u1ed1t nh\u1ea5t cho c\u00e1c m\u00f4 h\u00ecnh \u0111\u01b0\u1ee3c t\u1ed1i \u01b0u h\u00f3a v\u00e0 \u0111\u1ed9 ch\u00ednh x\u00e1c t\u1ed1i \u0111a.<\/p>\n<h2>Uber<\/h2>\n<p>Uber \u00e1p d\u1ee5ng h\u1ecdc s\u00e2u trong to\u00e0n b\u1ed9 ho\u1ea1t \u0111\u1ed9ng kinh doanh, t\u1eeb nghi\u00ean c\u1ee9u xe t\u1ef1 l\u00e1i \u0111\u1ebfn d\u1ef1 b\u00e1o h\u00e0nh tr\u00ecnh di chuy\u1ec3n v\u00e0 ng\u0103n ch\u1eb7n gian l\u1eadn.\u00a0Uber \u0111\u00e3 ph\u00e1t tri\u1ec3n\u00a0<a href=\"https:\/\/github.com\/horovod\/horovod\">Horovod<\/a>\u00a0, m\u1ed9t framework \u0111\u00e0o t\u1ea1o h\u1ecdc s\u00e2u ph\u00e2n t\u00e1n cho TensorFlow, Keras, PyTorch v\u00e0 Apache MXNet, \u0111\u1ec3 gi\u00fap t\u0103ng t\u1ed1c c\u00e1c d\u1ef1 \u00e1n h\u1ecdc s\u00e2u d\u1ec5 d\u00e0ng h\u01a1n v\u1edbi GPU v\u00e0 c\u00e1ch ti\u1ebfp c\u1eadn song song d\u1eef li\u1ec7u v\u1edbi \u0111\u00e0o t\u1ea1o ph\u00e2n t\u00e1n.\u00a0<a href=\"https:\/\/resources.nvidia.com\/gtcd-2020\/GTC2020-s21300?lx=3X9y6T\">Horvod hi\u1ec7n \u0111\u00e3 h\u1ed7 tr\u1ee3 Spark 3.0<\/a>\u00a0v\u1edbi t\u00ednh n\u0103ng l\u1eadp l\u1ecbch GPU v\u00e0 l\u1edbp KerasEstimator m\u1edbi s\u1eed d\u1ee5ng c\u00f4ng c\u1ee5 \u01b0\u1edbc t\u00ednh Spark v\u1edbi Spark ML Pipelines \u0111\u1ec3 t\u00edch h\u1ee3p t\u1ed1t h\u01a1n v\u00e0 d\u1ec5 s\u1eed d\u1ee5ng.\u00a0\u0110i\u1ec1u n\u00e0y cho ph\u00e9p c\u00e1c m\u00f4 h\u00ecnh TensorFlow v\u00e0 PyTorch \u0111\u01b0\u1ee3c \u0111\u00e0o t\u1ea1o tr\u1ef1c ti\u1ebfp tr\u00ean Spark DataFrames, t\u1eadn d\u1ee5ng kh\u1ea3 n\u0103ng c\u1ee7a Horovod \u0111\u1ec3 m\u1edf r\u1ed9ng song song t\u1edbi h\u00e0ng tr\u0103m GPU m\u00e0 kh\u00f4ng c\u1ea7n b\u1ea5t k\u1ef3 m\u00e3 chuy\u00ean d\u1ee5ng n\u00e0o \u0111\u1ec3 \u0111\u00e0o t\u1ea1o ph\u00e2n t\u00e1n.\u00a0V\u1edbi c\u00e1c API l\u1eadp l\u1ecbch v\u00e0 x\u1eed l\u00fd theo\u00a0c\u1ed9t nh\u1eadn bi\u1ebft tr\u00ecnh t\u0103ng t\u1ed1c m\u1edbi trong Apache Spark 3.0, c\u00f4ng vi\u1ec7c ETL s\u1ea3n xu\u1ea5t c\u00f3 th\u1ec3 chuy\u1ec3n d\u1eef li\u1ec7u cho Horovod \u0111\u1ec3 ch\u1ea1y ch\u01b0\u01a1ng tr\u00ecnh \u0111\u00e0o t\u1ea1o h\u1ecdc s\u00e2u ph\u00e2n t\u00e1n v\u1edbi GPU trong c\u00f9ng m\u1ed9t quy tr\u00ecnh.<\/p>\n<h2>T\u1ea1i sao Apache Spark l\u1ea1i quan tr\u1ecdng \u0111\u1ed1i v\u1edbi\u2026<\/h2>\n<p>Spark 3.0 \u0111\u00e1nh d\u1ea5u m\u1ed9t c\u1ed9t m\u1ed1c quan tr\u1ecdng \u0111\u1ec3 c\u00e1c nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u v\u00e0 k\u1ef9 s\u01b0 d\u1eef li\u1ec7u c\u1ed9ng t\u00e1c v\u1ec1 ph\u00e2n t\u00edch v\u00e0 AI, v\u00ec c\u00e1c ho\u1ea1t \u0111\u1ed9ng ETL hi\u1ec7n \u0111\u01b0\u1ee3c t\u0103ng t\u1ed1c trong khi c\u00e1c \u1ee9ng d\u1ee5ng ML v\u00e0 DL t\u1eadn d\u1ee5ng c\u00f9ng c\u01a1 s\u1edf h\u1ea1 t\u1ea7ng GPU.<\/p>\n<h2>C\u00e1c nh\u00f3m nghi\u00ean c\u1ee9u Khoa h\u1ecdc d\u1eef li\u1ec7u<\/h2>\n<p>S\u1ef1 k\u1ef3 di\u1ec7u c\u1ee7a khoa h\u1ecdc d\u1eef li\u1ec7u b\u1ecb c\u1ea3n tr\u1edf b\u1edfi nhi\u1ec1u nhi\u1ec7m v\u1ee5 \u0111\u01a1n gi\u1ea3n c\u1ea7n thi\u1ebft \u0111\u1ec3 s\u1eafp x\u1ebfp d\u1eef li\u1ec7u th\u00e0nh d\u1ea1ng c\u00f3 th\u1ec3 s\u1eed d\u1ee5ng \u0111\u01b0\u1ee3c.\u00a0Ph\u1ea7n l\u1edbn qu\u00e1 tr\u00ecnh \u0111\u00f3 li\u00ean quan \u0111\u1ebfn vi\u1ec7c s\u1eafp x\u1ebfp d\u1eef li\u1ec7u phi c\u1ea5u tr\u00fac nh\u01b0 m\u00e3 ZIP, ng\u00e0y th\u00e1ng v\u00e0 s\u1ed1 SKU tr\u00ean h\u00e0ng tri\u1ec7u ho\u1eb7c h\u00e0ng t\u1ef7 b\u1ea3n ghi.\u00a0T\u1eadp d\u1eef li\u1ec7u c\u00e0ng l\u1edbn th\u00ec qu\u00e1 tr\u00ecnh c\u00e0ng m\u1ea5t nhi\u1ec1u th\u1eddi gian.\u00a0Theo m\u1ed9t s\u1ed1 \u01b0\u1edbc t\u00ednh, vi\u1ec7c chu\u1ea9n b\u1ecb d\u1eef li\u1ec7u c\u00f3 th\u1ec3 ti\u00eau t\u1ed1n 80% th\u1eddi gian c\u1ee7a nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u.<\/p>\n<p>Hadoop l\u00e0 m\u1ed9t c\u00f4ng ngh\u1ec7 \u0111\u1ed9t ph\u00e1 \u0111\u1ec3 th\u1ef1c hi\u1ec7n ph\u00e2n t\u00edch d\u1eef li\u1ec7u tr\u00ean quy m\u00f4 l\u1edbn, gi\u00fap c\u00e1c nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u c\u00f3 th\u1ec3 th\u1ef1c hi\u1ec7n c\u00e1c truy v\u1ea5n \u0111\u1ed1i v\u1edbi c\u00e1c kho d\u1eef li\u1ec7u r\u1ea5t l\u1edbn.\u00a0Tuy nhi\u00ean, th\u1eddi gian x\u1eed l\u00fd d\u00e0i, \u0111\u1eb7c bi\u1ec7t khi c\u1ea7n qu\u00e9t l\u1eb7p l\u1ea1i tr\u00ean t\u1eadp d\u1eef li\u1ec7u hi\u1ec7n c\u00f3, nh\u01b0 tr\u01b0\u1eddng h\u1ee3p th\u01b0\u1eddng x\u1ea3y ra trong vi\u1ec7c s\u1eafp x\u1ebfp v\u00e0 kh\u00e1m ph\u00e1 d\u1eef li\u1ec7u.<\/p>\n<p>Spark \u0111\u01b0\u1ee3c x\u00e2y d\u1ef1ng c\u00f3 m\u1ee5c \u0111\u00edch cho c\u00e1c truy v\u1ea5n l\u1eb7p l\u1ea1i tr\u00ean c\u00e1c t\u1eadp d\u1eef li\u1ec7u l\u1edbn.\u00a0V\u1edbi t\u1ed1c \u0111\u1ed9 nhanh h\u01a1n t\u1edbi 100 l\u1ea7n so v\u1edbi Hadoop\/MapReduce, ngay l\u1eadp t\u1ee9c g\u00e2y \u1ea5n t\u01b0\u1ee3ng v\u1edbi c\u00e1c nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u.\u00a0Spark c\u0169ng c\u00f3 th\u1ec3 d\u1ec5 d\u00e0ng \u0111\u00e1p \u1ee9ng c\u00e1c ng\u00f4n ng\u1eef ph\u00e1t tri\u1ec3n theo \u0111\u1ecbnh h\u01b0\u1edbng khoa h\u1ecdc d\u1eef li\u1ec7u nh\u01b0 Python, R v\u00e0 Scala.\u00a0B\u1edfi v\u00ec h\u1ea7u h\u1ebft c\u00e1c nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u th\u00edch l\u00e0m vi\u1ec7c v\u1edbi m\u1ed9t c\u00f4ng c\u1ee5 l\u1eadp tr\u00ecnh duy nh\u1ea5t n\u00ean Spark c\u00f3 th\u1ec3 d\u1ec5 d\u00e0ng th\u00edch \u1ee9ng v\u1edbi nhu c\u1ea7u c\u1ee7a t\u1eebng ng\u01b0\u1eddi d\u00f9ng.<\/p>\n<p>Spark SQL gi\u1edbi thi\u1ec7u m\u1ed9t kh\u00e1i ni\u1ec7m tr\u1eebu t\u01b0\u1ee3ng h\u00f3a d\u1eef li\u1ec7u \u0111\u01b0\u1ee3c g\u1ecdi l\u00e0 DataFrames h\u1ed7 tr\u1ee3 c\u1ea3 d\u1eef li\u1ec7u c\u00f3 c\u1ea5u tr\u00fac v\u00e0 phi c\u1ea5u tr\u00fac c\u00f3 th\u1ec3 \u0111\u01b0\u1ee3c thao t\u00e1c b\u1eb1ng nhi\u1ec1u ng\u00f4n ng\u1eef kh\u00e1c nhau.\u00a0N\u00f3 cho ph\u00e9p ng\u00f4n ng\u1eef SQL quen thu\u1ed9c \u0111\u01b0\u1ee3c \u00e1p d\u1ee5ng cho d\u1eef li\u1ec7u phi c\u1ea5u tr\u00fac theo nh\u1eefng c\u00e1ch ch\u01b0a t\u1eebng c\u00f3 tr\u01b0\u1edbc \u0111\u00e2y.\u00a0Spark ML cung c\u1ea5p m\u1ed9t b\u1ed9 API c\u1ea5p cao, \u0111\u01b0\u1ee3c x\u00e2y d\u1ef1ng d\u1ef1a tr\u00ean DataFrames \u0111\u1ec3 x\u00e2y d\u1ef1ng c\u00e1c quy tr\u00ecnh ho\u1eb7c quy tr\u00ecnh c\u00f4ng vi\u1ec7c ML.\u00a0\u0110i\u1ec1u n\u00e0y cung c\u1ea5p kh\u1ea3 n\u0103ng m\u1edf r\u1ed9ng x\u1eed l\u00fd d\u1eef li\u1ec7u \u0111\u01b0\u1ee3c ph\u00e2n v\u00f9ng v\u1edbi s\u1ef1 d\u1ec5 d\u00e0ng c\u1ee7a SQL \u0111\u1ec3 thao t\u00e1c d\u1eef li\u1ec7u.<\/p>\n<h2>\u0110\u1ed9i ng\u0169 k\u1ef9 s\u01b0 d\u1eef li\u1ec7u<\/h2>\n<p>C\u00e1c k\u1ef9 s\u01b0 d\u1eef li\u1ec7u thu h\u1eb9p kho\u1ea3ng c\u00e1ch gi\u1eefa c\u00e1c nh\u00e0 khoa h\u1ecdc v\u00e0 nh\u00e0 ph\u00e1t tri\u1ec3n d\u1eef li\u1ec7u.\u00a0Trong khi c\u00e1c nh\u00e0 khoa h\u1ecdc d\u1eef li\u1ec7u ch\u1ecdn lo\u1ea1i d\u1eef li\u1ec7u v\u00e0 thu\u1eadt to\u00e1n ph\u00f9 h\u1ee3p \u0111\u1ec3 gi\u1ea3i quy\u1ebft v\u1ea5n \u0111\u1ec1 th\u00ec c\u00e1c k\u1ef9 s\u01b0 d\u1eef li\u1ec7u l\u00e0m vi\u1ec7c v\u1edbi c\u00e1c nh\u00e0 khoa h\u1ecdc v\u00e0 nh\u00e0 ph\u00e1t tri\u1ec3n d\u1eef li\u1ec7u v\u1ec1 m\u1ecdi th\u1ee9 li\u00ean quan \u0111\u1ebfn vi\u1ec7c t\u1ea1o \u0111\u01b0\u1eddng d\u1eabn d\u1eef li\u1ec7u \u0111\u1ec3 chuy\u1ec3n \u0111\u1ed5i, l\u01b0u tr\u1eef v\u00e0 ph\u00e2n t\u00edch tr\u00edch xu\u1ea5t d\u1eef li\u1ec7u nh\u1eb1m x\u00e2y d\u1ef1ng c\u00e1c \u1ee9ng d\u1ee5ng ph\u00e2n t\u00edch d\u1eef li\u1ec7u l\u1edbn.<\/p>\n<p>Spark lo\u1ea1i b\u1ecf s\u1ef1 ph\u1ee9c t\u1ea1p kh\u1ecfi ph\u01b0\u01a1ng tr\u00ecnh l\u01b0u tr\u1eef.\u00a0V\u00ec framework n\u00e0y c\u00f3 th\u1ec3 ho\u1ea1t \u0111\u1ed9ng v\u1edbi h\u1ea7u h\u1ebft m\u1ecdi b\u1ed9 l\u01b0u tr\u1eef c\u01a1 b\u1ea3n, bao g\u1ed3m h\u1ec7 th\u1ed1ng t\u1ec7p ph\u00e2n t\u00e1n Hadoop, n\u00ean \u0111\u00e2y l\u00e0 m\u1ed9t framework linh ho\u1ea1t h\u01a1n Hadoop v\u00e0 c\u00f3 kh\u1ea3 n\u0103ng th\u00edch \u1ee9ng cao h\u01a1n v\u1edbi s\u1ef1 k\u1ebft h\u1ee3p gi\u1eefa c\u01a1 s\u1edf h\u1ea1 t\u1ea7ng \u0111\u00e1m m\u00e2y v\u00e0 t\u1ea1i ch\u1ed7.\u00a0Spark c\u0169ng c\u00f3 th\u1ec3 d\u1ec5 d\u00e0ng k\u1ebft h\u1ee3p c\u00e1c ngu\u1ed3n d\u1eef li\u1ec7u ph\u00e1t tr\u1ef1c tuy\u1ebfn, bi\u1ebfn n\u00f3 th\u00e0nh m\u1ed9t c\u00f4ng c\u1ee5 th\u00edch h\u1ee3p cho th\u1ebf h\u1ec7 \u1ee9ng d\u1ee5ng internet v\u1ea1n v\u1eadt.<\/p>\n<div>____<br \/><b>B\u00e0i vi\u1ebft li\u00ean quan<\/b><\/div>\n<ul>\n<li><a href=\"https:\/\/thegioimaychu.vn\/blog\/?post_type=post&#038;p=23415\">B\u01b0\u1edbc v\u00e0o k\u1ef7 nguy\u00ean tr\u1ee3 l\u00fd AI c\u00e1 nh\u00e2n &#8211; so s\u00e1nh chi ti\u1ebft NVIDIA RTX Spark v\u00e0 DGX Spark<\/a><\/li>\n<li><a href=\"https:\/\/thegioimaychu.vn\/blog\/?post_type=post&#038;p=23322\">NVIDIA RTX PRO 4500 Blackwell Server Edition: B\u01b0\u1edbc nh\u1ea3y v\u1ecdt t\u1eeb th\u1ebf h\u1ec7 L4 cho h\u1ea1 t\u1ea7ng AI suy lu\u1eadn hi\u1ec7u n\u0103ng cao<\/a><\/li>\n<li><a href=\"https:\/\/thegioimaychu.vn\/blog\/?post_type=post&#038;p=23119\">NVIDIA NGC\u2011ready Low\u2011latency Edge AI: Gi\u1ea3i ph\u00e1p cho Retail, Manufacturing v\u00e0 Smart Cities<\/a><\/li>\n<li><a href=\"https:\/\/thegioimaychu.vn\/blog\/?post_type=post&#038;p=23110\">So s\u00e1nh s\u1ee9c m\u1ea1nh c\u1ee7a m\u00e1y t\u00ednh AI DGX Spark v\u1edbi c\u00e1c card GPU m\u00e1y tr\u1ea1m chuy\u00ean nghi\u1ec7p c\u1ee7a NVIDIA<\/a><\/li>\n<li><a href=\"https:\/\/thegioimaychu.vn\/blog\/?post_type=post&#038;p=22969\">T\u1eeb Orin \u0111\u1ebfn Thor: B\u01b0\u1edbc nh\u1ea3y v\u1ecdt ki\u1ebfn tr\u00fac c\u1ee7a NVIDIA JetPack 7.0 v\u00e0 t\u01b0\u01a1ng lai c\u1ee7a robot h\u00ecnh ng\u01b0\u1eddi<\/a><\/li>\n<li><a href=\"https:\/\/thegioimaychu.vn\/blog\/?post_type=post&#038;p=22887\">T\u1ea1i sao g\u1ecdi NVIDIA DGX Spark l\u00e0 Si\u00eau m\u00e1y t\u00ednh AI c\u00e1 nh\u00e2n?<\/a><\/li>\n<\/ul>\n<\/p>","protected":false},"excerpt":{"rendered":"<p>Apache Spark l&agrave; m&#7897;t framework m&atilde; ngu&#7891;n m&#7903; &#273;&#7875; x&#7917; l&yacute; song song c&aacute;c t&aacute;c v&#7909; d&#7919; li&#7879;u l&#7899;n tr&ecirc;n c&aacute;c h&#7879; th&#7889;ng &#273;&#432;&#7907;c ph&acirc;n c&#7909;m.&nbsp;&#272;&acirc;y l&agrave; m&#7897;t trong nh&#7919;ng framework x&#7917; l&yacute; ph&acirc;n t&aacute;n &#273;&#432;&#7907;c s&#7917; d&#7909;ng r&#7897;ng r&atilde;i nh&#7845;t tr&ecirc;n th&#7871; gi&#7899;i. Apache Spark l&agrave; g&igrave;? Song song v&#7899;i s&#7921; gia t&#259;ng v&#432;&#7907;t&#8230;<\/p>\n","protected":false},"author":29,"featured_media":17546,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[3],"tags":[7,12,29,1314],"class_list":["post-17538","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-hpc","tag-ai","tag-deep-learning","tag-machine-learning","tag-nvidia"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark - Blog | TheGioiMayChu<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark - Blog | TheGioiMayChu\" \/>\n<meta name=\"twitter:description\" content=\"Apache Spark l&agrave; m&#7897;t framework m&atilde; ngu&#7891;n m&#7903; &#273;&#7875; x&#7917; l&yacute; song song c&aacute;c t&aacute;c v&#7909; d&#7919; li&#7879;u l&#7899;n tr&ecirc;n c&aacute;c h&#7879; th&#7889;ng &#273;&#432;&#7907;c ph&acirc;n c&#7909;m.&nbsp;&#272;&acirc;y l&agrave; m&#7897;t trong nh&#7919;ng framework x&#7917; l&yacute; ph&acirc;n t&aacute;n &#273;&#432;&#7907;c s&#7917; d&#7909;ng r&#7897;ng r&atilde;i nh&#7845;t tr&ecirc;n th&#7871; gi&#7899;i. Apache Spark l&agrave; g&igrave;? Song song v&#7899;i s&#7921; gia t&#259;ng v&#432;&#7907;t...\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/TheGioiMayChu-Blog-Image.jpg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TGMC Blog\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"24 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/\"},\"author\":{\"name\":\"TGMC Blog\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#\\\/schema\\\/person\\\/777f41403962330933a4fbcc80bec570\"},\"headline\":\"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark\",\"datePublished\":\"2023-09-15T01:04:45+00:00\",\"dateModified\":\"2023-09-15T01:11:05+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/\"},\"wordCount\":4784,\"publisher\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/09\\\/TheGioiMayChu-Blog-Image.jpg\",\"keywords\":[\"ai\",\"deep learning\",\"machine learning\",\"NVIDIA\"],\"articleSection\":[\"AI - HPC\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/\",\"url\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/\",\"name\":\"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark - Blog | TheGioiMayChu\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/09\\\/TheGioiMayChu-Blog-Image.jpg\",\"datePublished\":\"2023-09-15T01:04:45+00:00\",\"dateModified\":\"2023-09-15T01:11:05+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/#primaryimage\",\"url\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/09\\\/TheGioiMayChu-Blog-Image.jpg\",\"contentUrl\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/09\\\/TheGioiMayChu-Blog-Image.jpg\",\"width\":600,\"height\":400},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/ai-hpc\\\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Trang ch\u1ee7\",\"item\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI - HPC\",\"item\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/.\\\/ai-hpc\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/\",\"name\":\"Blog | TheGioiMayChu\",\"description\":\"C\u1eadp nh\u1eadt xu h\u01b0\u1edbng c\u00f4ng ngh\u1ec7 m\u00e1y ch\u1ee7\",\"publisher\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#organization\",\"name\":\"Blog | TheGioiMayChu\",\"url\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/08\\\/cropped-ntcnlog-logo-2.png\",\"contentUrl\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/08\\\/cropped-ntcnlog-logo-2.png\",\"width\":500,\"height\":73,\"caption\":\"Blog | TheGioiMayChu\"},\"image\":{\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/thegioimaychu\\\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/#\\\/schema\\\/person\\\/777f41403962330933a4fbcc80bec570\",\"name\":\"TGMC Blog\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ff179df3aeac078da5e07e40f2e014e99ab4120e2276b687bbe7e6964d99655e?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ff179df3aeac078da5e07e40f2e014e99ab4120e2276b687bbe7e6964d99655e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ff179df3aeac078da5e07e40f2e014e99ab4120e2276b687bbe7e6964d99655e?s=96&d=mm&r=g\",\"caption\":\"TGMC Blog\"},\"url\":\"https:\\\/\\\/thegioimaychu.vn\\\/blog\\\/author\\\/tgmc\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark - Blog | TheGioiMayChu","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/","twitter_card":"summary_large_image","twitter_title":"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark - Blog | TheGioiMayChu","twitter_description":"Apache Spark l&agrave; m&#7897;t framework m&atilde; ngu&#7891;n m&#7903; &#273;&#7875; x&#7917; l&yacute; song song c&aacute;c t&aacute;c v&#7909; d&#7919; li&#7879;u l&#7899;n tr&ecirc;n c&aacute;c h&#7879; th&#7889;ng &#273;&#432;&#7907;c ph&acirc;n c&#7909;m.&nbsp;&#272;&acirc;y l&agrave; m&#7897;t trong nh&#7919;ng framework x&#7917; l&yacute; ph&acirc;n t&aacute;n &#273;&#432;&#7907;c s&#7917; d&#7909;ng r&#7897;ng r&atilde;i nh&#7845;t tr&ecirc;n th&#7871; gi&#7899;i. Apache Spark l&agrave; g&igrave;? Song song v&#7899;i s&#7921; gia t&#259;ng v&#432;&#7907;t...","twitter_image":"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/TheGioiMayChu-Blog-Image.jpg","twitter_misc":{"Written by":"TGMC Blog","Est. reading time":"24 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/#article","isPartOf":{"@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/"},"author":{"name":"TGMC Blog","@id":"https:\/\/thegioimaychu.vn\/blog\/#\/schema\/person\/777f41403962330933a4fbcc80bec570"},"headline":"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark","datePublished":"2023-09-15T01:04:45+00:00","dateModified":"2023-09-15T01:11:05+00:00","mainEntityOfPage":{"@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/"},"wordCount":4784,"publisher":{"@id":"https:\/\/thegioimaychu.vn\/blog\/#organization"},"image":{"@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/#primaryimage"},"thumbnailUrl":"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/TheGioiMayChu-Blog-Image.jpg","keywords":["ai","deep learning","machine learning","NVIDIA"],"articleSection":["AI - HPC"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/","url":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/","name":"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark - Blog | TheGioiMayChu","isPartOf":{"@id":"https:\/\/thegioimaychu.vn\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/#primaryimage"},"image":{"@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/#primaryimage"},"thumbnailUrl":"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/TheGioiMayChu-Blog-Image.jpg","datePublished":"2023-09-15T01:04:45+00:00","dateModified":"2023-09-15T01:11:05+00:00","breadcrumb":{"@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/#primaryimage","url":"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/TheGioiMayChu-Blog-Image.jpg","contentUrl":"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2023\/09\/TheGioiMayChu-Blog-Image.jpg","width":600,"height":400},{"@type":"BreadcrumbList","@id":"https:\/\/thegioimaychu.vn\/blog\/ai-hpc\/apache-spark-la-gi-gioi-thieu-tong-quan-ve-apache-spark-p17538\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Trang ch\u1ee7","item":"https:\/\/thegioimaychu.vn\/blog\/"},{"@type":"ListItem","position":2,"name":"AI - HPC","item":"https:\/\/thegioimaychu.vn\/blog\/.\/ai-hpc\/"},{"@type":"ListItem","position":3,"name":"Apache Spark l\u00e0 g\u00ec? Gi\u1edbi thi\u1ec7u t\u1ed5ng quan v\u1ec1 Apache Spark"}]},{"@type":"WebSite","@id":"https:\/\/thegioimaychu.vn\/blog\/#website","url":"https:\/\/thegioimaychu.vn\/blog\/","name":"Blog | TheGioiMayChu","description":"C\u1eadp nh\u1eadt xu h\u01b0\u1edbng c\u00f4ng ngh\u1ec7 m\u00e1y ch\u1ee7","publisher":{"@id":"https:\/\/thegioimaychu.vn\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/thegioimaychu.vn\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/thegioimaychu.vn\/blog\/#organization","name":"Blog | TheGioiMayChu","url":"https:\/\/thegioimaychu.vn\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/thegioimaychu.vn\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2018\/08\/cropped-ntcnlog-logo-2.png","contentUrl":"https:\/\/thegioimaychu.vn\/blog\/wp-content\/uploads\/2018\/08\/cropped-ntcnlog-logo-2.png","width":500,"height":73,"caption":"Blog | TheGioiMayChu"},"image":{"@id":"https:\/\/thegioimaychu.vn\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/thegioimaychu\/"]},{"@type":"Person","@id":"https:\/\/thegioimaychu.vn\/blog\/#\/schema\/person\/777f41403962330933a4fbcc80bec570","name":"TGMC Blog","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/ff179df3aeac078da5e07e40f2e014e99ab4120e2276b687bbe7e6964d99655e?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/ff179df3aeac078da5e07e40f2e014e99ab4120e2276b687bbe7e6964d99655e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ff179df3aeac078da5e07e40f2e014e99ab4120e2276b687bbe7e6964d99655e?s=96&d=mm&r=g","caption":"TGMC Blog"},"url":"https:\/\/thegioimaychu.vn\/blog\/author\/tgmc\/"}]}},"_links":{"self":[{"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/posts\/17538","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/comments?post=17538"}],"version-history":[{"count":8,"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/posts\/17538\/revisions"}],"predecessor-version":[{"id":17595,"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/posts\/17538\/revisions\/17595"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/media\/17546"}],"wp:attachment":[{"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/media?parent=17538"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/categories?post=17538"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/thegioimaychu.vn\/blog\/wp-json\/wp\/v2\/tags?post=17538"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}