Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Exam Dumps

Databricks Certified Associate Developer for Apache Spark 3.0 Exam

Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Sample Questions

Which of the following code blocks immediately removes the previously cached DataFrame transactionsDf from memory and disk?

 A. array_remove(transactionsDf, "*") 
B. transactionsDf.unpersist() (Correct)
 C. del transactionsDf 
D. transactionsDf.clearCache() E. transactionsDf.persist()

The code block shown below should return a new 2-column DataFrame that shows one attribute from column attributes per row next to the associated itemName, for all suppliers in column supplier whose name includes Sports. Choose the answer that correctly fills the blanks in the code block to accomplish this. Sample of DataFrame itemsDf: 1. +------+----------------------------------+-----------------------------+-------------------+ 2. |itemId|itemName |attributes |supplier | 3. +------+----------------------------------+-----------------------------+-------------------+ 4. |1 |Thick Coat for Walking in the Snow|[blue, winter, cozy] |Sports Company Inc.| 5. |2 |Elegant Outdoors Summer Dress |[red, summer, fresh, cooling]|YetiX | 6. |3 |Outdoors Backpack |[green, summer, travel] |Sports Company Inc.| 7. +------+----------------------------------+-----------------------------+-------------------+ Code block: itemsDf.__1__(__2__).select(__3__, __4__) 

A. 1. filter 2. col("supplier").isin("Sports") 3. "itemName" 4. explode(col("attributes")) 
B. 1. where 2. col("supplier").contains("Sports") 3. "itemName" 4. "attributes" 
C. 1. where 2. col(supplier).contains("Sports") 3. explode(attributes) 4. itemName
 D. 1. where 2. "Sports".isin(col("Supplier")) 3. "itemName" 4. array_explode("attributes") 
E. 1. filter 2. col("supplier").contains("Sports") 3. "itemName" 4. explode("attributes")

The code block shown below should add a column itemNameBetweenSeparators to DataFrame itemsDf. The column should contain arrays of maximum 4 strings. The arrays should be composed of the values in column itemsDf which are separated at - or whitespace characters. Choose the answer that correctly fills the blanks in the code block to accomplish this. Sample of DataFrame itemsDf: 1. +------+----------------------------------+-------------------+ 2. |itemId|itemName |supplier | 3. +------+----------------------------------+-------------------+ 4. |1 |Thick Coat for Walking in the Snow|Sports Company Inc.| 5. |2 |Elegant Outdoors Summer Dress |YetiX | 6. |3 |Outdoors Backpack |Sports Company Inc.| 7. +------+----------------------------------+-------------------+ Code block: itemsDf.__1__(__2__, __3__(__4__, "[\s\-]", __5__)) 

A. 1. withColumn 2. "itemNameBetweenSeparators" 3. split 4. "itemName" 5. 4 (Correct) 
B. 1. withColumnRenamed 2. "itemNameBetweenSeparators" 3. split 4. "itemName" 5. 4 
C. 1. withColumnRenamed 2. "itemName" 3. split 4. "itemNameBetweenSeparators" 5. 4 
D. 1. withColumn 2. "itemNameBetweenSeparators" 3. split 4. "itemName" 5. 5 
E. 1. withColumn 2. itemNameBetweenSeparators 3. str_split 4. "itemName" 5. 5 

Which of the following code blocks reads in the two-partition parquet file stored at filePath, making sure all columns are included exactly once even though each partition has a different schema? Schema of first partition: 1. root 2. |-- transactionId: integer (nullable = true) 3. |-- predError: integer (nullable = true) 4. |-- value: integer (nullable = true) 5. |-- storeId: integer (nullable = true) 6. |-- productId: integer (nullable = true) 7. |-- f: integer (nullable = true) Schema of second partition: 1. root 2. |-- transactionId: integer (nullable = true) 3. |-- predError: integer (nullable = true) 4. |-- value: integer (nullable = true) 5. |-- storeId: integer (nullable = true) 6. |-- rollId: integer (nullable = true) 7. |-- f: integer (nullable = true) 8. |-- tax_id: integer (nullable = false) 

A. spark.read.parquet(filePath, mergeSchema='y') 
B. spark.read.option("mergeSchema", "true").parquet(filePath) 
C. spark.read.parquet(filePath) 
D. 1. nx = 0 2. for file in dbutils.fs.ls(filePath): 3. if not file.name.endswith(".parquet"): 4. continue 5. df_temp = spark.read.parquet(file.path) 6. if nx == 0: 7. df = df_temp 8. else: 9. df = df.union(df_temp) 10. nx = nx+1 11. df E. 1. nx = 0 2. for file in dbutils.fs.ls(filePath): 3. if not file.name.endswith(".parquet"): 4. continue 5. df_temp = spark.read.parquet(file.path) 6. if nx == 0: 7. df = df_temp 8. else: 9. df = df.join(df_temp, how="outer") 10. nx = nx+1 11. df 

Which of the following code blocks shows the structure of a DataFrame in a tree-like way, containing both column names and types? 

A. 1. print(itemsDf.columns) 2. print(itemsDf.types) 
B. itemsDf.printSchema() 
C. spark.schema(itemsDf) 
D. itemsDf.rdd.printSchema()
 E. itemsDf.print.schema() 

The code block shown below should add column transactionDateForm to DataFrame transactionsDf. The column should express the unix-format timestamps in column transactionDate as string type like Apr 26 (Sunday). Choose the answer that correctly fills the blanks in the code block to accomplish this. transactionsDf.__1__(__2__, from_unixtime(__3__, __4__)) 

A. 1. withColumn 2. "transactionDateForm" 3. "MMM d (EEEE)" 4. "transactionDate" 
B. 1. select 2. "transactionDate" 3. "transactionDateForm" 4. "MMM d (EEEE)" 
C. 1. withColumn 2. "transactionDateForm" 3. "transactionDate" 4. "MMM d (EEEE)" 
D. 1. withColumn 2. "transactionDateForm" 3. "transactionDate" 4. "MM d (EEE)" 
E. 1. withColumnRenamed 2. "transactionDate" 3. "transactionDateForm" 4. "MM d (EEE)" 

Which of the following code blocks reads in the JSON file stored at filePath as a DataFrame? 

A. spark.read.json(filePath) 
B. spark.read.path(filePath, source="json") 
C. spark.read().path(filePath) 
D. spark.read().json(filePath) 
E. spark.read.path(filePath) 

The code block displayed below contains an error. The code block should write DataFrame transactionsDf as a parquet file to location filePath after partitioning it on column storeId. Find the error. Code block: transactionsDf.write.partitionOn("storeId").parquet(filePath) 

A. The partitioning column as well as the file path should be passed to the write() method of DataFrame transactionsDf directly and not as appended commands as in the code block. 
B. The partitionOn method should be called before the write method.
 C. The operator should use the mode() option to configure the DataFrameWriter so that it replaces any existing files at location filePath. 
D. Column storeId should be wrapped in a col() operator. 
E. No method partitionOn() exists for the DataFrame class, partitionBy() should be used instead. 

Which of the following code blocks creates a new DataFrame with 3 columns, productId, highest, and lowest, that shows the biggest and smallest values of column value per value in column productId from DataFrame transactionsDf? Sample of DataFrame transactionsDf: 1. +-------------+---------+-----+-------+---------+----+ 2. |transactionId|predError|value|storeId|productId| f| 3. +-------------+---------+-----+-------+---------+----+ 4. | 1| 3| 4| 25| 1|null| 5. | 2| 6| 7| 2| 2|null| 6. | 3| 3| null| 25| 3|null| 7. | 4| null| null| 3| 2|null| 8. | 5| null| null| null| 2|null| 9. | 6| 3| 2| 25| 2|null| 10. +-------------+---------+-----+-------+---------+----+ 

A. transactionsDf.max('value').min('value') 
B. transactionsDf.agg(max('value').alias('highest'), min('value').alias('lowest')) 
C. transactionsDf.groupby(col(productId)).agg(max(col(value)).alias("highest"), min(col(value)).alias("lowest"))
 D. transactionsDf.groupby('productId').agg(max('value').alias('highest'), min('value').alias('lowest'))
 E. transactionsDf.groupby("productId").agg({"highest": max("value"), "lowest": min("value")}) 

Which of the following code blocks returns a DataFrame with approximately 1,000 rows from the 10,000-row DataFrame itemsDf, without any duplicates, returning the same rows even if the code block is run twice? 

A. itemsDf.sampleBy("row", fractions={0: 0.1}, seed=82371) 
B. itemsDf.sample(fraction=0.1, seed=87238) 
C. itemsDf.sample(fraction=1000, seed=98263) 
D. itemsDf.sample(withReplacement=True, fraction=0.1, seed=23536) 
E. itemsDf.sample(fraction=0.1) 

Which of the following code blocks returns a DataFrame where columns predError and productId areremoved from DataFrame transactionsDf?Sample of DataFrame transactionsDf:1. +-------------+---------+-----+-------+---------+----+2. |transactionId|predError|value|storeId|productId|f |3. +-------------+---------+-----+-------+---------+----+4. |1 |3 |4 |25 |1 |null|5. |2 |6 |7 |2 |2 |null|6. |3 |3 |null |25 |3 |null|7. +-------------+---------+-----+-------+---------+----+

A.transactionsDf.withColumnRemoved("predError", "productId")
B. transactionsDf.drop(["predError", "productId", "associateId"])
C. transactionsDf.drop("predError", "productId", "associateId")
D. transactionsDf.dropColumns("predError", "productId", "associateId")
E. transactionsDf.drop(col("predError", "productId"))

Which of the following code blocks returns about 150 randomly selected rows from the 1000-rowDataFrame transactionsDf, assuming that any row can appear more than once in the returnedDataFrame?

A. transactionsDf.resample(0.15, False, 3142)
B. transactionsDf.sample(0.15, False, 3142)
C. transactionsDf.sample(0.15)
D. transactionsDf.sample(0.85, 8429)
E. transactionsDf.sample(True, 0.15, 8261)

The code block displayed below contains an error. The code block should use Python methodfind_most_freq_letter to find the letter present most in column itemName of DataFrame itemsDfandreturn it in a new column most_frequent_letter. Find the error.Code block:1. find_most_freq_letter_udf = udf(find_most_freq_letter)2. itemsDf.withColumn("most_frequent_letter", find_most_freq_letter("itemName"))

A. Spark is not using the UDF method correctly.
B. The UDF method is not registered correctly, since the return type is missing.
C. The "itemName" expression should be wrapped in col().
D. UDFs do not exist in PySpark.
E. Spark is not adding a column.