admin HYDERABAD, India 4 Questions 7 Answers 1 Best Answer 27 Points View Profile 6 admin Beginner Asked: January 7, 2020In: Technology What is GC Overhead Limit Exceeded Hadoop 6 Hi All, When i tried to list of files in my hdfs folder by using hdfs fs -ls /myfolder command i am facing GC Overhead Limit exceeded exception error in hadoop.Please help me how to solve this error. big datagc overhead limithadoop Share Facebook 2 Answers Voted Oldest Recent Anonymous Added an answer on January 7, 2020 at 4:39 pm This answer was edited. Hello Srilakshmi, GC Overhead Limit exception error will come when doing data copy from local to hdfs or hdfs to local and setup a replication for hdfs files at that time also this GC overhead limit exceeded error will come. We need to setup HADOOP_CLIENT_OPTS fix this error. export HADOOP_CLIENT_OPTS=”-XX:-UseGCOverheadLimit -Xmx4096m” HADOOP_CLIENT_OPTS – All hadoop components run as a client. This was needed because all the hadoop commands run as a client. 3 Reply Share Share Share on Facebook Share on Twitter Share on LinkedIn Share on WhatsApp srilakshmi 5 Questions 7 Answers 1 Best Answer 47 Points View Profile srilakshmi Beginner Added an answer on January 7, 2020 at 4:30 pm Hello , Generally GC Overhead limit exceeded error will come in below senories. If the hdfs folder having more then 1 million files and if you tried to run hdfs commands like -ls,-du -h,-copyToLocal etc. How to solve GC Overhead limit exceeded error Hadoop commands we run on edgenode so increase hadoop client heap memory on fly . Open hadoop edge node in putty Login to your account copy paste below command on your terminal to solve GC Overhead limit exceeded error in your hadoop environment. export HADOOP_CLIENT_OPTS=”$HADOOP_OPTS -Xmx5g” The above command will use full to fix GC Overhead limit exceeded error in hadoop and 5g indicates heap memory you can increase the value based on your requirement. 3 Reply Share Share Share on Facebook Share on Twitter Share on LinkedIn Share on WhatsApp Leave an answerLeave an answerCancel reply Attachment Select file Browse Featured image Select file Browse Add a video to describe the problem better. Video type Youtube Vimeo Dialymotion Facebook Choose from here the video type. Video ID Put here the video id : https://www.youtube.com/watch?v=sdUUx5FdySs Ex: 'sdUUx5FdySs'. What is the capital of Egypt? ( Cairo ) Save my name, email, and website in this browser for the next time I comment.