快捷導(dǎo)航

Presto自定義函數(shù)@SqlNullable引發(fā)問(wèn)題詳解

更新時(shí)間：2022年12月05日 11:39:21 作者：EdurtIO

這篇文章主要為大家介紹了Presto自定義函數(shù)@SqlNullable引發(fā)問(wèn)題詳解，有需要的朋友可以借鑒參考下，希望能夠有所幫助，祝大家多多進(jìn)步，早日升職加薪

引發(fā)Presto問(wèn)題

看到標(biāo)題我們會(huì)想到是由于@SqlNullable注解引發(fā)的問(wèn)題，我們先看一段代碼，正是這段有意思的代碼，讓我糾結(jié)了2個(gè)多小時(shí)，引發(fā)了Presto的問(wèn)題。

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
public static Slice userId(@SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}

這段代碼很簡(jiǎn)單，就是我們將傳遞進(jìn)來(lái)的base64的字符串解碼成實(shí)際的字符串，單單從代碼上看是不會(huì)有什么問(wèn)題的。當(dāng)我們實(shí)際運(yùn)行這個(gè)函數(shù)的時(shí)候問(wèn)題就出現(xiàn)了，以下是一個(gè)使用示例：

select user_id(str) from temp.users limit 100;

看執(zhí)行的SQL似乎也沒(méi)有什么異常，但就是這么簡(jiǎn)簡(jiǎn)單單的一個(gè)函數(shù)引發(fā)了Presto的問(wèn)題，那就是java.lang.NullPointerException: undefined錯(cuò)誤

錯(cuò)誤具體內(nèi)容

java.lang.NullPointerException: undefined
	at io.prestosql.type.VarcharOperators.equal(VarcharOperators.java:53)
	at io.prestosql.$gen.CursorProcessor_20200927_063218_2398.filter(Unknown Source)
	at io.prestosql.$gen.CursorProcessor_20200927_063218_2398.process(Unknown Source)
	at io.prestosql.operator.ScanFilterAndProjectOperator$RecordCursorToPages.process(ScanFilterAndProjectOperator.java:323)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:181)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.lambda$flatten$6(WorkProcessorUtils.java:277)
	at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:319)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$finishWhen$3(WorkProcessorUtils.java:215)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:149)
	at io.prestosql.operator.Driver.processInternal(Driver.java:379)
	at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
	at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
	at io.prestosql.operator.Driver.processFor(Driver.java:276)
	at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)
	at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:shichengoooo@163.com)
	at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
	at io.prestosql.$gen.Presto_341____20200925_110330_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

修改UDF源碼

通過(guò)上面錯(cuò)誤我們推算出應(yīng)該是由于數(shù)據(jù)導(dǎo)致空指針異常，那么問(wèn)題出在哪里呢？問(wèn)題就出在查詢(xún)str這個(gè)字段中，這個(gè)字段我們經(jīng)過(guò)實(shí)際的查詢(xún)發(fā)現(xiàn)是有''的數(shù)據(jù)，在做轉(zhuǎn)換的時(shí)候出現(xiàn)了空指針問(wèn)題，于是我們修改UDF的源碼為

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
@SqlNullable
public static Slice userId(@SqlNullable @SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}

我們?cè)诜椒ê蛥?shù)上添加了@SqlNullable注解用于標(biāo)記此函數(shù)可以接收空的數(shù)據(jù)，這似乎看著也沒(méi)有問(wèn)題，我們將該函數(shù)重新發(fā)布，再次執(zhí)行SQL發(fā)現(xiàn)還存在相同的問(wèn)題，于是又將代碼修改為以下內(nèi)容

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
public static Slice userId(@SqlNullable @SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}

刪除了方法上的@SqlNullable注解，再次運(yùn)行發(fā)現(xiàn)不會(huì)再出現(xiàn)這個(gè)錯(cuò)誤，但是Presto服務(wù)中不斷的報(bào)出空指針錯(cuò)誤，只是不在反饋給查詢(xún)客戶(hù)端，原本以為此問(wèn)題已經(jīng)解決，然而更有意思的事情發(fā)生了，我們使用了343版本測(cè)試成功后，線上版本是341，升級(jí)線上后發(fā)現(xiàn)此問(wèn)題再次復(fù)現(xiàn)，如果再次在方法上加上@SqlNullable注解在341版本上又會(huì)修復(fù)這個(gè)問(wèn)題，目前已將這個(gè)問(wèn)題反饋給官方，推薦大家使用343版本！

以上就是Presto自定義函數(shù)@SqlNullable引發(fā)問(wèn)題詳解的詳細(xì)內(nèi)容，更多關(guān)于Presto自定義函數(shù)@SqlNullable的資料請(qǐng)關(guān)注腳本之家其它相關(guān)文章！

您可能感興趣的文章: