Jump to content

Help : hive query

vendettaa

By vendettaa
in Discussions

 Share

Recommended Posts

vendettaa Sr. Hero

vendettaa

Posted
  • Author
Posted
23 minutes ago, NPReddy said:

Not sure buddy. Since you asked for spark, i just tried to help you. How do u write hive queries? In shell? I think, You can create some functions to performa this. Since you are dealing with billions of records, i do not think it is recommended. Wait for experts.

Udf but don't want to go for it 

Link to comment
Share on other sites
vendettaa Sr. Hero

vendettaa

Posted
  • Author
Posted
2 hours ago, NPReddy said:

Im not an expert but I think you can use except data frame api to perform this.

put table 1 data into data frame 1

table 2 data into another data frame 2.

dataframe1.select(keyColumn).except.dataframe2.select(keycolumn)

you will get data from dataframe 1 which is not present in df2. May not be a perfect answer but you can change it according to your use case.

@NPReddy  thank you so much man 

except is helpful

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...