问题描述
我有以下火花数据框:
val df = Seq(("US",10),("IND",20),("NZ",30),("CAN",40)).toDF("a","b")
df.show(false)
+---+---+
|a |b |
+---+---+
|US |10 |
|IND|20 |
|NZ |30 |
|CAN|40 |
+---+---+
我正在应用 when()
条件,如下所示:
df.withColumn("x",when(col("a").isin(us_list:_*),"u").when(col("a").isin(i_list:_*),"i").when(col("a").isin(n_list:_*),"n").otherwise("-")).show(false)
+---+---+---+
|a |b |x |
+---+---+---+
|US |10 |u |
|IND|20 |i |
|NZ |30 |n |
|CAN|40 |- |
+---+---+---+
现在为了最小化代码,我正在尝试以下操作:
val us_list = Array("U","US")
val i_list = Array("I","IND")
val n_list = Array("N","NZ")
val ar1 = Array((us_list,"u"),(i_list,"i"),(n_list,"n"))
val ap = ar1.map( x => when(col("a").isInCollection(x._1),x._2) )
结果是
ap: Array[org.apache.spark.sql.Column] = Array(CASE WHEN (a IN (U,US)) THEN u END,CASE WHEN (a IN (I,IND)) THEN i END,CASE WHEN (a IN (N,NZ)) THEN n END)
但是当我尝试
val ap = ar1.map( x => when(col("a").isInCollection(x._1),x._2) ).reduce( (x,y) => x.y )
解决方法
您可以在 using Excel1 = Microsoft.Office.Interop.Excel;
Excel1.Application xlApp;
Excel1.Workbook xlWorkBook;
Excel1._Worksheet xlWorkSheet;
xlApp = new Excel1.Application();
xlWorkBook = xlApp.Workbooks.Add();
xlWorkSheet = (Excel1._Worksheet)xlWorkBook.ActiveSheet;
Excel1.Style style = xlWorkBook.Styles.Add("NewStyle");
style.Font.Name = "Courier New";
for (i = 0; i < lines.Length; i++)
{
Excel1.Range rangeStyles = xlWorkSheet.get_Range("A:XFD");
rangeStyles.Style = "NewStyle";
cells = lines[i].Split(new Char[] { '\t',';' });
for (j = 0; j < cells.Length; j++)
xlWorkSheet.Cells[i + 1,j + 1] = cells[j];
}
列表上使用 foldLeft :
npm install --save jquery
,
通常不需要使用 when
/reduce
等组合 fold
语句。coalesce
就足够了,因为 when
语句是按顺序计算的,并在条件为假时给出 null
。它还可以让您免于指定 otherwise
,因为您只需将一列附加到 coalesce
的参数列表中。
val ar1 = Array((us_list,"u"),(i_list,"i"),(n_list,"n"))
val ap = ar1.map( x => when(col("a").isInCollection(x._1),x._2) )
val combined = coalesce(ap :+ lit("-"): _*)
df.withColumn("x",combined).show
+---+---+---+
| a| b| x|
+---+---+---+
| US| 10| u|
|IND| 20| i|
| NZ| 30| n|
|CAN| 40| -|
+---+---+---+