Pig Script

Posts: 8
Joined: Fri Dec 30, 2016 10:50 am

Pig Script

Postby anantajb » Fri Aug 18, 2017 4:17 am

When executing a pig script, I want to delete a folder in hdfs.

I tried with
sh hdfs dfs -rm -r '/user/anantajb/brandTop';
Since the folder does not exist, the script fails.
I want to delete the folder if it exists.....

I tried
sh "hdfs dfs -test -d '/user/anantajb/brandTop' ; if [ \\$? -eq 0 ]; then -rm -r '/user/anantajb/brandTop'; else echo 'Directory Not Found'; fi"
which was one of the solutions I found on stackoverflow.... but it does not work... system throws syntax error.

https://stackoverflow.com/questions/256 ... pig-script

Please suggest a solution......

Finance Junkie
Posts: 946
Joined: Wed Apr 09, 2014 6:28 am

Re: Pig Script

Postby edupristine » Mon Aug 21, 2017 10:25 am

no need to give path in single quotes. remove quotes and try

Delete a file “abc.txt" from the "examples" directory.

hdfs dfs -rm /user/venkata/data/examples/abc.txt

Delete all files from the "example1" directory using a wildcard.

hdfs dfs -rm /user/venkata/data/examples/example1/*

Remove the entire "example" directory and all of its contents in HDFS.

hdfs dfs -rm -r /user/venkata/data/examples

Return to “Big Data & Hadoop”


Global Association of Risk Professionals, Inc. (GARP®) does not endorse, promote, review or warrant the accuracy of the products or services offered by EduPristine for FRM® related information, nor does it endorse any pass rates claimed by the provider. Further, GARP® is not responsible for any fees or costs paid by the user to EduPristine nor is GARP® responsible for any fees or costs of any person or entity providing any services to EduPristine Study Program. FRM®, GARP® and Global Association of Risk Professionals®, are trademarks owned by the Global Association of Risk Professionals, Inc

CFA Institute does not endorse, promote, or warrant the accuracy or quality of the products or services offered by EduPristine. CFA Institute, CFA®, Claritas® and Chartered Financial Analyst® are trademarks owned by CFA Institute.

Utmost care has been taken to ensure that there is no copyright violation or infringement in any of our content. Still, in case you feel that there is any copyright violation of any kind please send a mail to abuse@edupristine.com and we will rectify it.