Getting "was not completed in time" error when preprocessing dataset

[AftabHussain]

Hi I am getting a preprocessing error (when invoking source preprocess.sh). I don't get any error when I preprocess the same dataset with code2vec. Appreciate any advice. Here's the error:

Extracting paths from training set...
dir: <dataset dir> was not completed in time
dir: <dataset dir> was not completed in time
dir: <dataset dir> was not completed in time
dir: <dataset dir> was not completed in time
Finished extracting paths from training set
Creating histograms from the training data
subtoken vocab size:  0
node vocab size:  0
target vocab size:  0
File: <dataset_name>.raw.txt
Traceback (most recent call last):
  File "preprocess.py", line 115, in <module>
    max_contexts=int(args.max_contexts), max_data_contexts=int(args.max_data_contexts))
  File "preprocess.py", line 53, in process_file
    print('Average total contexts: ' + str(float(sum_total) / total))
ZeroDivisionError: float division by zero

This is the line that is being triggered:

code2seq/JavaExtractor/extract.py

Line 49 in af04b4c

print('dir: ' + str(dir) + ' was not completed in time', file=sys.stderr)

Appreciate any thoughts.

[urialon]

Hi @AftabHussain ,
Thank you for your interest in code2seq!

The fact that the error includes <dataset dir> might hint that you entered <dataset dir> somewhere in the preprocess.sh script.

Can you verify your paths?

[AftabHussain]

Hi @urialon ,

My pleasure, thank you for your message. Yes I entered the directory path in the preprocess.sh script, and verified the paths. I had entered the same paths for the preprocess.sh script in code2vec, and did not have any issues there. But with code2sec, we are getting this time limit error message.

[urialon]

Hi @AftabHussain ,
Can you please try running the Java process directly for a single file:

java -cp JavaExtractor/JPredict/target/JavaExtractor-0.0.1-SNAPSHOT.jar JavaExtractor.App --max_path_length=8 --max_path_width=2 --file JavaExtractor/JPredict/src/main/java/JavaExtractor/App.java

Uri

[sadiasahar]

Hi @urialon ,
I am also getting same preprocessing error. I tried to run Java process directly for a single file as you say and got the following output.

main void,Void0|Mth|Nm1,METHOD_NAME void,Void0|Mth|Prm|VDID0,args void,Void0|Mth|Prm|Cls1,string void,Void0|Mth|Prm|ArBr2,[] METHOD_NAME,Nm1|Mth|Prm|VDID0,args METHOD_NAME,Nm1|Mth|Prm|Cls1,string METHOD_NAME,Nm1|Mth|Prm|ArBr2,[] METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|As0|Nm0,s|command|line|values METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|As0|ObjEx|Cls0,command|line|values METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|As0|ObjEx|Nm1,args METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Prm|VDID0,e METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Prm|Cls1,cmd|line|exception METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm0,e METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace METHOD_NAME,Nm1|Mth|Bk|If|Neq|Fld0|Nm0,s|command|line|values METHOD_NAME,Nm1|Mth|Bk|If|Neq|Fld0|Nm2,file METHOD_NAME,Nm1|Mth|Bk|If|Neq|Null1,null METHOD_NAME,Nm1|Mth|Bk|If|Bk|Ex|VDE|Cls0,extract|features|task METHOD_NAME,Nm1|Mth|Bk|If|Bk|Ex|VDE|VD|VDID0,extract|features|task METHOD_NAME,Nm1|Mth|Bk|If|Bk|Ex|Cal0|Nm0,extract|features|task METHOD_NAME,Nm1|Mth|Bk|If|Bk|Ex|Cal0|Nm2,process|file METHOD_NAME,Nm1|Mth|Bk|If|If|Neq|Fld0|Nm0,s|command|line|values METHOD_NAME,Nm1|Mth|Bk|If|If|Neq|Fld0|Nm2,dir METHOD_NAME,Nm1|Mth|Bk|If|If|Neq|Null1,null METHOD_NAME,Nm1|Mth|Bk|If|If|Bk|Ex|Cal0|Nm1,extract|dir args,VDID0|Prm|Cls1,string args,VDID0|Prm|ArBr2,[] args,VDID0|Prm|Mth|Bk|Try|Bk|Ex|As0|Nm0,s|command|line|values args,VDID0|Prm|Mth|Bk|Try|Catch|Prm|VDID0,e args,VDID0|Prm|Mth|Bk|Try|Catch|Prm|Cls1,cmd|line|exception args,VDID0|Prm|Mth|Bk|If|Neq|Fld0|Nm0,s|command|line|values args,VDID0|Prm|Mth|Bk|If|Neq|Fld0|Nm2,file args,VDID0|Prm|Mth|Bk|If|Neq|Null1,null args,VDID0|Prm|Mth|Bk|If|Bk|Ex|VDE|Cls0,extract|features|task args,VDID0|Prm|Mth|Bk|If|Bk|Ex|Cal0|Nm0,extract|features|task args,VDID0|Prm|Mth|Bk|If|Bk|Ex|Cal0|Nm2,process|file args,VDID0|Prm|Mth|Bk|If|If|Neq|Fld0|Nm0,s|command|line|values args,VDID0|Prm|Mth|Bk|If|If|Neq|Fld0|Nm2,dir args,VDID0|Prm|Mth|Bk|If|If|Neq|Null1,null string,Cls1|Prm|ArBr2,[] string,Cls1|Prm|Mth|Bk|Try|Bk|Ex|As0|Nm0,s|command|line|values string,Cls1|Prm|Mth|Bk|Try|Catch|Prm|VDID0,e string,Cls1|Prm|Mth|Bk|Try|Catch|Prm|Cls1,cmd|line|exception string,Cls1|Prm|Mth|Bk|If|Neq|Fld0|Nm0,s|command|line|values string,Cls1|Prm|Mth|Bk|If|Neq|Fld0|Nm2,file string,Cls1|Prm|Mth|Bk|If|Neq|Null1,null string,Cls1|Prm|Mth|Bk|If|Bk|Ex|VDE|Cls0,extract|features|task string,Cls1|Prm|Mth|Bk|If|Bk|Ex|Cal0|Nm0,extract|features|task string,Cls1|Prm|Mth|Bk|If|Bk|Ex|Cal0|Nm2,process|file string,Cls1|Prm|Mth|Bk|If|If|Neq|Fld0|Nm0,s|command|line|values string,Cls1|Prm|Mth|Bk|If|If|Neq|Fld0|Nm2,dir string,Cls1|Prm|Mth|Bk|If|If|Neq|Null1,null [],ArBr2|Prm|Mth|Bk|Try|Bk|Ex|As0|Nm0,s|command|line|values [],ArBr2|Prm|Mth|Bk|Try|Catch|Prm|VDID0,e [],ArBr2|Prm|Mth|Bk|Try|Catch|Prm|Cls1,cmd|line|exception [],ArBr2|Prm|Mth|Bk|If|Neq|Fld0|Nm0,s|command|line|values [],ArBr2|Prm|Mth|Bk|If|Neq|Fld0|Nm2,file [],ArBr2|Prm|Mth|Bk|If|Neq|Null1,null [],ArBr2|Prm|Mth|Bk|If|Bk|Ex|VDE|Cls0,extract|features|task [],ArBr2|Prm|Mth|Bk|If|Bk|Ex|Cal0|Nm0,extract|features|task [],ArBr2|Prm|Mth|Bk|If|Bk|Ex|Cal0|Nm2,process|file [],ArBr2|Prm|Mth|Bk|If|If|Neq|Fld0|Nm0,s|command|line|values [],ArBr2|Prm|Mth|Bk|If|If|Neq|Fld0|Nm2,dir [],ArBr2|Prm|Mth|Bk|If|If|Neq|Null1,null s|command|line|values,Nm0|As|ObjEx|Cls0,command|line|values s|command|line|values,Nm0|As|ObjEx|Nm1,args s|command|line|values,Nm0|As|Ex|Bk|Try|Catch|Prm|VDID0,e s|command|line|values,Nm0|As|Ex|Bk|Try|Catch|Prm|Cls1,cmd|line|exception s|command|line|values,Nm0|As|Ex|Bk|Try|Bk|If|Neq|Null1,null command|line|values,Cls0|ObjEx1|Nm1,args command|line|values,Cls0|ObjEx1|As|Ex|Bk|Try|Catch|Prm|VDID0,e command|line|values,Cls0|ObjEx1|As|Ex|Bk|Try|Catch|Prm|Cls1,cmd|line|exception args,Nm1|ObjEx1|As|Ex|Bk|Try|Catch|Prm|VDID0,e args,Nm1|ObjEx1|As|Ex|Bk|Try|Catch|Prm|Cls1,cmd|line|exception e,VDID0|Prm|Cls1,cmd|line|exception e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm0,e e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace e,VDID0|Prm|Catch|Try|Bk|If|Neq|Fld0|Nm0,s|command|line|values e,VDID0|Prm|Catch|Try|Bk|If|Neq|Fld0|Nm2,file e,VDID0|Prm|Catch|Try|Bk|If|Neq|Null1,null e,VDID0|Prm|Catch|Try|Bk|If|If|Neq|Null1,null cmd|line|exception,Cls1|Prm|Catch|Bk|Ex|Cal0|Nm0,e cmd|line|exception,Cls1|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace cmd|line|exception,Cls1|Prm|Catch|Try|Bk|If|Neq|Fld0|Nm0,s|command|line|values cmd|line|exception,Cls1|Prm|Catch|Try|Bk|If|Neq|Fld0|Nm2,file cmd|line|exception,Cls1|Prm|Catch|Try|Bk|If|Neq|Null1,null cmd|line|exception,Cls1|Prm|Catch|Try|Bk|If|If|Neq|Null1,null e,Nm0|Cal|Nm2,print|stack|trace s|command|line|values,Nm0|Fld|Nm2,file s|command|line|values,Nm0|Fld|Neq|Null1,null s|command|line|values,Nm0|Fld|Neq|If|Bk|Ex|VDE|Cls0,extract|features|task s|command|line|values,Nm0|Fld|Neq|If|Bk|Ex|VDE|VD|VDID0,extract|features|task s|command|line|values,Nm0|Fld|Neq|If|Bk|Ex|Cal0|Nm0,extract|features|task s|command|line|values,Nm0|Fld|Neq|If|Bk|Ex|Cal0|Nm2,process|file s|command|line|values,Nm0|Fld|Neq|If|If|Neq|Fld0|Nm0,s|command|line|values s|command|line|values,Nm0|Fld|Neq|If|If|Neq|Fld0|Nm2,dir s|command|line|values,Nm0|Fld|Neq|If|If|Neq|Null1,null s|command|line|values,Nm0|Fld|Neq|If|If|Bk|Ex|Cal0|Nm1,extract|dir file,Nm2|Fld|Neq|Null1,null file,Nm2|Fld|Neq|If|Bk|Ex|VDE|Cls0,extract|features|task file,Nm2|Fld|Neq|If|Bk|Ex|VDE|VD|VDID0,extract|features|task file,Nm2|Fld|Neq|If|Bk|Ex|Cal0|Nm0,extract|features|task file,Nm2|Fld|Neq|If|Bk|Ex|Cal0|Nm2,process|file file,Nm2|Fld|Neq|If|If|Neq|Fld0|Nm0,s|command|line|values file,Nm2|Fld|Neq|If|If|Neq|Fld0|Nm2,dir file,Nm2|Fld|Neq|If|If|Neq|Null1,null file,Nm2|Fld|Neq|If|If|Bk|Ex|Cal0|Nm1,extract|dir null,Null1|Neq|If|Bk|Ex|VDE|Cls0,extract|features|task null,Null1|Neq|If|Bk|Ex|VDE|VD|VDID0,extract|features|task null,Null1|Neq|If|Bk|Ex|VDE|VD|ObjEx|Cls0,extract|features|task null,Null1|Neq|If|Bk|Ex|VDE|VD|ObjEx|Nm1,s|command|line|values null,Null1|Neq|If|Bk|Ex|Cal0|Nm0,extract|features|task null,Null1|Neq|If|Bk|Ex|Cal0|Nm2,process|file null,Null1|Neq|If|If|Neq|Fld0|Nm0,s|command|line|values null,Null1|Neq|If|If|Neq|Fld0|Nm2,dir null,Null1|Neq|If|If|Neq|Null1,null null,Null1|Neq|If|If|Bk|Ex|Cal0|Nm1,extract|dir extract|features|task,Cls0|VDE|VD|VDID0,extract|features|task extract|features|task,Cls0|VDE|VD|ObjEx|Cls0,extract|features|task extract|features|task,Cls0|VDE|VD|ObjEx|Nm1,s|command|line|values extract|features|task,Cls0|VDE|VD|ObjEx|Cal2|Fld0|Nm0,s|command|line|values extract|features|task,Cls0|VDE|VD|ObjEx|Cal2|Fld0|Nm2,file extract|features|task,Cls0|VDE|VD|ObjEx|Cal2|Nm2,to|path extract|features|task,Cls0|VDE|Ex|Bk|Ex|Cal0|Nm0,extract|features|task extract|features|task,Cls0|VDE|Ex|Bk|Ex|Cal0|Nm2,process|file extract|features|task,Cls0|VDE|Ex|Bk|If|If|Neq|Fld0|Nm0,s|command|line|values extract|features|task,Cls0|VDE|Ex|Bk|If|If|Neq|Fld0|Nm2,dir extract|features|task,Cls0|VDE|Ex|Bk|If|If|Neq|Null1,null extract|features|task,VDID0|VD|ObjEx|Cls0,extract|features|task extract|features|task,VDID0|VD|ObjEx|Nm1,s|command|line|values extract|features|task,VDID0|VD|ObjEx|Cal2|Fld0|Nm0,s|command|line|values extract|features|task,VDID0|VD|ObjEx|Cal2|Fld0|Nm2,file extract|features|task,VDID0|VD|ObjEx|Cal2|Nm2,to|path extract|features|task,VDID0|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,extract|features|task extract|features|task,VDID0|VD|VDE|Ex|Bk|Ex|Cal0|Nm2,process|file extract|features|task,VDID0|VD|VDE|Ex|Bk|If|If|Neq|Null1,null extract|features|task,Cls0|ObjEx|Nm1,s|command|line|values extract|features|task,Cls0|ObjEx|Cal2|Fld0|Nm0,s|command|line|values extract|features|task,Cls0|ObjEx|Cal2|Fld0|Nm2,file extract|features|task,Cls0|ObjEx|Cal2|Nm2,to|path extract|features|task,Cls0|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,extract|features|task extract|features|task,Cls0|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm2,process|file s|command|line|values,Nm1|ObjEx|Cal2|Fld0|Nm0,s|command|line|values s|command|line|values,Nm1|ObjEx|Cal2|Fld0|Nm2,file s|command|line|values,Nm1|ObjEx|Cal2|Nm2,to|path s|command|line|values,Nm1|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,extract|features|task s|command|line|values,Nm1|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm2,process|file s|command|line|values,Nm0|Fld0|Nm2,file s|command|line|values,Nm0|Fld0|Cal|Nm2,to|path file,Nm2|Fld0|Cal|Nm2,to|path extract|features|task,Nm0|Cal|Nm2,process|file extract|features|task,Nm0|Cal|Ex|Bk|If|If|Neq|Fld0|Nm0,s|command|line|values extract|features|task,Nm0|Cal|Ex|Bk|If|If|Neq|Fld0|Nm2,dir extract|features|task,Nm0|Cal|Ex|Bk|If|If|Neq|Null1,null process|file,Nm2|Cal|Ex|Bk|If|If|Neq|Fld0|Nm0,s|command|line|values process|file,Nm2|Cal|Ex|Bk|If|If|Neq|Fld0|Nm2,dir process|file,Nm2|Cal|Ex|Bk|If|If|Neq|Null1,null s|command|line|values,Nm0|Fld|Nm2,dir s|command|line|values,Nm0|Fld|Neq|Null1,null s|command|line|values,Nm0|Fld|Neq|If|Bk|Ex|Cal0|Nm1,extract|dir dir,Nm2|Fld|Neq|Null1,null dir,Nm2|Fld|Neq|If|Bk|Ex|Cal0|Nm1,extract|dir null,Null1|Neq|If|Bk|Ex|Cal0|Nm1,extract|dir
extract|dir void,Void0|Mth|Nm1,METHOD_NAME void,Void0|Mth|Bk|Ex|VDE|Cls0,thread|pool|executor void,Void0|Mth|Bk|Ex|VDE|VD|VDID0,executor void,Void0|Mth|Bk|Ex|VDE|VD|Cast|Cls0,thread|pool|executor void,Void0|Mth|Bk|Ex|VDE|VD|Cast|Cal1|Nm0,executors void,Void0|Mth|Bk|Ex|VDE|VD|Cast|Cal1|Nm3,new|fixed|thread|pool void,Void0|Mth|Bk|Ex|VDE|Cls|Cls0,extract|features|task void,Void0|Mth|Bk|Ex|VDE|VD|VDID0,tasks void,Void0|Mth|Bk|Ex|VDE|VD|ObjEx|Cls0,linked|list void,Void0|Mth|Bk|Try|Bk|Ex|Cal0|Cal0|Nm3,filter void,Void0|Mth|Bk|Try|Bk|Ex|Cal0|Nm3,for|each void,Void0|Mth|Bk|Try|Catch|Prm|VDID0,e void,Void0|Mth|Bk|Try|Catch|Prm|Cls1,io|exception void,Void0|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm0,e void,Void0|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace void,Void0|Mth|Bk|Ex|VDE|Cls|Cls|Cls0,void void,Void0|Mth|Bk|Ex|VDE|VD|VDID0,tasks|results void,Void0|Mth|Bk|Ex|VDE|VD|Null1,null void,Void0|Mth|Bk|Try|Bk|Ex|As0|Nm0,tasks|results void,Void0|Mth|Bk|Try|Bk|Ex|As0|Cal1|Nm0,executor void,Void0|Mth|Bk|Try|Bk|Ex|As0|Cal1|Nm2,tasks void,Void0|Mth|Bk|Try|Bk|Ex|As0|Cal1|Nm3,invoke|all void,Void0|Mth|Bk|Try|Catch|Prm|VDID0,e void,Void0|Mth|Bk|Try|Catch|Prm|Cls1,interrupted|exception void,Void0|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm0,e void,Void0|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace void,Void0|Mth|Bk|Try|Bk|Ex|Cal0|Nm0,executor void,Void0|Mth|Bk|Try|Bk|Ex|Cal0|Nm2,shutdown void,Void0|Mth|Bk|Ex|Cal0|Nm0,tasks|results void,Void0|Mth|Bk|Ex|Cal0|Lambda|Prm|VDID0,f void,Void0|Mth|Bk|Ex|Cal0|Nm3,for|each METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|Cls0,thread|pool|executor METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|VDID0,executor METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|Cast|Cls0,thread|pool|executor METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|Cast|Cal1|Nm0,executors METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|Cast|Cal1|Nm3,new|fixed|thread|pool METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|Cls|Cls0,extract|features|task METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|VDID0,tasks METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|ObjEx|Cls0,linked|list METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|Cal0|Cal0|Nm3,filter METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|Cal0|Nm3,for|each METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Prm|VDID0,e METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Prm|Cls1,io|exception METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm0,e METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|Cls|Cls|Cls0,void METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|VDID0,tasks|results METHOD_NAME,Nm1|Mth|Bk|Ex|VDE|VD|Null1,null METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|As0|Nm0,tasks|results METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|As0|Cal1|Nm0,executor METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|As0|Cal1|Nm2,tasks METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|As0|Cal1|Nm3,invoke|all METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Prm|VDID0,e METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Prm|Cls1,interrupted|exception METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm0,e METHOD_NAME,Nm1|Mth|Bk|Try|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|Cal0|Nm0,executor METHOD_NAME,Nm1|Mth|Bk|Try|Bk|Ex|Cal0|Nm2,shutdown METHOD_NAME,Nm1|Mth|Bk|Ex|Cal0|Nm0,tasks|results METHOD_NAME,Nm1|Mth|Bk|Ex|Cal0|Lambda|Prm|VDID0,f METHOD_NAME,Nm1|Mth|Bk|Ex|Cal0|Nm3,for|each thread|pool|executor,Cls0|VDE|VD|VDID0,executor thread|pool|executor,Cls0|VDE|VD|Cast|Cls0,thread|pool|executor thread|pool|executor,Cls0|VDE|VD|Cast|Cal1|Nm0,executors thread|pool|executor,Cls0|VDE|VD|Cast|Cal1|Fld2|Nm0,s|command|line|values thread|pool|executor,Cls0|VDE|VD|Cast|Cal1|Fld2|Nm2,num|threads thread|pool|executor,Cls0|VDE|VD|Cast|Cal1|Nm3,new|fixed|thread|pool thread|pool|executor,Cls0|VDE|Ex|Bk|Ex|VDE|Cls|Cls0,extract|features|task thread|pool|executor,Cls0|VDE|Ex|Bk|Ex|VDE|VD|VDID0,tasks thread|pool|executor,Cls0|VDE|Ex|Bk|Ex|VDE|VD|ObjEx|Cls0,linked|list thread|pool|executor,Cls0|VDE|Ex|Bk|Try|Bk|Ex|Cal0|Nm3,for|each thread|pool|executor,Cls0|VDE|Ex|Bk|Try|Catch|Prm|VDID0,e thread|pool|executor,Cls0|VDE|Ex|Bk|Try|Catch|Prm|Cls1,io|exception executor,VDID0|VD|Cast|Cls0,thread|pool|executor executor,VDID0|VD|Cast|Cal1|Nm0,executors executor,VDID0|VD|Cast|Cal1|Fld2|Nm0,s|command|line|values executor,VDID0|VD|Cast|Cal1|Fld2|Nm2,num|threads executor,VDID0|VD|Cast|Cal1|Nm3,new|fixed|thread|pool executor,VDID0|VD|VDE|Ex|Bk|Ex|VDE|Cls|Cls0,extract|features|task executor,VDID0|VD|VDE|Ex|Bk|Ex|VDE|VD|VDID0,tasks executor,VDID0|VD|VDE|Ex|Bk|Try|Catch|Prm|VDID0,e executor,VDID0|VD|VDE|Ex|Bk|Try|Catch|Prm|Cls1,io|exception thread|pool|executor,Cls0|Cast|Cal1|Nm0,executors thread|pool|executor,Cls0|Cast|Cal1|Fld2|Nm0,s|command|line|values thread|pool|executor,Cls0|Cast|Cal1|Fld2|Nm2,num|threads thread|pool|executor,Cls0|Cast|Cal1|Nm3,new|fixed|thread|pool executors,Nm0|Cal|Fld2|Nm0,s|command|line|values executors,Nm0|Cal|Fld2|Nm2,num|threads s|command|line|values,Nm0|Fld2|Nm2,num|threads s|command|line|values,Nm0|Fld2|Cal|Nm3,new|fixed|thread|pool num|threads,Nm2|Fld2|Cal|Nm3,new|fixed|thread|pool extract|features|task,Cls0|Cls|VDE|VD|VDID0,tasks extract|features|task,Cls0|Cls|VDE|VD|ObjEx|Cls0,linked|list extract|features|task,Cls0|Cls|VDE|Ex|Bk|Try|Catch|Prm|VDID0,e extract|features|task,Cls0|Cls|VDE|Ex|Bk|Try|Catch|Prm|Cls1,io|exception extract|features|task,Cls0|Cls|VDE|Ex|Bk|Ex|VDE|VD|VDID0,tasks|results extract|features|task,Cls0|Cls|VDE|Ex|Bk|Ex|VDE|VD|Null1,null tasks,VDID0|VD|ObjEx|Cls0,linked|list tasks,VDID0|VD|VDE|Ex|Bk|Try|Catch|Prm|VDID0,e tasks,VDID0|VD|VDE|Ex|Bk|Try|Catch|Prm|Cls1,io|exception tasks,VDID0|VD|VDE|Ex|Bk|Ex|VDE|VD|VDID0,tasks|results tasks,VDID0|VD|VDE|Ex|Bk|Ex|VDE|VD|Null1,null files,Nm0|Cal0|Cal2|Nm0,paths files,Nm0|Cal0|Cal2|Fld2|Nm0,s|command|line|values files,Nm0|Cal0|Cal2|Fld2|Nm2,dir files,Nm0|Cal0|Cal2|Nm3,get files,Nm0|Cal0|Cal0|MethRef|Type|Cls0,files files,Nm0|Cal0|Cal0|Cal0|Lambda|Prm|VDID0,p files,Nm0|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|Cal0|Nm2,to|lower|case files,Nm0|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|StrEx2,java files,Nm0|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|Nm3,ends|with files,Nm0|Cal0|Cal0|Cal0|Cal|Lambda|Prm|VDID0,f paths,Nm0|Cal2|Fld2|Nm0,s|command|line|values paths,Nm0|Cal2|Fld2|Nm2,dir paths,Nm0|Cal2|Cal0|Nm3,walk paths,Nm0|Cal2|Cal0|Cal0|MethRef|Type|Cls0,files paths,Nm0|Cal2|Cal0|Cal0|Cal0|Lambda|Prm|VDID0,p paths,Nm0|Cal2|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|StrEx2,java paths,Nm0|Cal2|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|Nm3,ends|with paths,Nm0|Cal2|Cal0|Cal0|Cal0|Cal|Lambda|Prm|VDID0,f s|command|line|values,Nm0|Fld2|Nm2,dir s|command|line|values,Nm0|Fld2|Cal2|Nm3,get s|command|line|values,Nm0|Fld2|Cal2|Cal0|Nm3,walk s|command|line|values,Nm0|Fld2|Cal2|Cal0|Cal0|MethRef|Type|Cls0,files s|command|line|values,Nm0|Fld2|Cal2|Cal0|Cal0|Cal0|Lambda|Prm|VDID0,p dir,Nm2|Fld2|Cal2|Nm3,get dir,Nm2|Fld2|Cal2|Cal0|Nm3,walk dir,Nm2|Fld2|Cal2|Cal0|Cal0|MethRef|Type|Cls0,files dir,Nm2|Fld2|Cal2|Cal0|Cal0|Cal0|Lambda|Prm|VDID0,p get,Nm3|Cal2|Cal0|Nm3,walk get,Nm3|Cal2|Cal0|Cal0|MethRef|Type|Cls0,files get,Nm3|Cal2|Cal0|Cal0|Cal0|Lambda|Prm|VDID0,p get,Nm3|Cal2|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|StrEx2,java get,Nm3|Cal2|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|Nm3,ends|with get,Nm3|Cal2|Cal0|Cal0|Cal0|Cal|Lambda|Prm|VDID0,f walk,Nm3|Cal0|Cal0|MethRef|Type|Cls0,files walk,Nm3|Cal0|Cal0|Cal0|Lambda|Prm|VDID0,p walk,Nm3|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|Cal0|Nm2,to|lower|case walk,Nm3|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|StrEx2,java walk,Nm3|Cal0|Cal0|Cal0|Lambda|Ex|Cal0|Nm3,ends|with walk,Nm3|Cal0|Cal0|Cal0|Cal|Lambda|Prm|VDID0,f files,Cls0|Type|MethRef2|Cal0|Nm3,filter files,Cls0|Type|MethRef2|Cal0|Cal0|Lambda|Prm|VDID0,p files,Cls0|Type|MethRef2|Cal0|Cal0|Lambda|Ex|Cal0|StrEx2,java files,Cls0|Type|MethRef2|Cal0|Cal0|Lambda|Ex|Cal0|Nm3,ends|with files,Cls0|Type|MethRef2|Cal0|Cal0|Cal|Lambda|Prm|VDID0,f filter,Nm3|Cal0|Cal0|Lambda|Prm|VDID0,p filter,Nm3|Cal0|Cal0|Lambda|Ex|Cal0|Cal0|Cal0|Nm0,p filter,Nm3|Cal0|Cal0|Lambda|Ex|Cal0|Cal0|Cal0|Nm2,to|string filter,Nm3|Cal0|Cal0|Lambda|Ex|Cal0|Cal0|Nm2,to|lower|case filter,Nm3|Cal0|Cal0|Lambda|Ex|Cal0|StrEx2,java filter,Nm3|Cal0|Cal0|Lambda|Ex|Cal0|Nm3,ends|with filter,Nm3|Cal0|Cal0|Cal|Lambda|Prm|VDID0,f filter,Nm3|Cal0|Cal0|Cal|Lambda|Bk|Ex|VDE|Cls0,extract|features|task filter,Nm3|Cal0|Cal0|Cal|Lambda|Bk|Ex|Cal0|Nm0,tasks filter,Nm3|Cal0|Cal0|Cal|Lambda|Bk|Ex|Cal0|Nm2,task filter,Nm3|Cal0|Cal0|Cal|Lambda|Bk|Ex|Cal0|Nm3,add p,VDID0|Prm|Lambda2|Ex|Cal0|Cal0|Cal0|Nm0,p p,VDID0|Prm|Lambda2|Ex|Cal0|Cal0|Cal0|Nm2,to|string p,VDID0|Prm|Lambda2|Ex|Cal0|Cal0|Nm2,to|lower|case p,VDID0|Prm|Lambda2|Ex|Cal0|StrEx2,java p,VDID0|Prm|Lambda2|Ex|Cal0|Nm3,ends|with p,VDID0|Prm|Lambda2|Cal0|Nm3,filter p,VDID0|Prm|Lambda2|Cal0|Cal|Lambda|Prm|VDID0,f p,Nm0|Cal0|Nm2,to|string p,Nm0|Cal0|Cal0|Nm2,to|lower|case p,Nm0|Cal0|Cal0|Cal|StrEx2,java p,Nm0|Cal0|Cal0|Cal|Ex|Lambda2|Cal0|Nm3,filter to|string,Nm2|Cal0|Cal0|Nm2,to|lower|case to|string,Nm2|Cal0|Cal0|Cal|StrEx2,java to|string,Nm2|Cal0|Cal0|Cal|Ex|Lambda2|Cal0|Nm3,filter to|lower|case,Nm2|Cal0|Cal|StrEx2,java to|lower|case,Nm2|Cal0|Cal|Ex|Lambda2|Cal0|Nm3,filter java,StrEx2|Cal|Nm3,ends|with java,StrEx2|Cal|Ex|Lambda2|Cal0|Nm3,filter java,StrEx2|Cal|Ex|Lambda2|Cal0|Cal|Lambda|Prm|VDID0,f ends|with,Nm3|Cal|Ex|Lambda2|Cal0|Nm3,filter ends|with,Nm3|Cal|Ex|Lambda2|Cal0|Cal|Lambda|Prm|VDID0,f filter,Nm3|Cal0|Cal|Lambda|Prm|VDID0,f filter,Nm3|Cal0|Cal|Lambda|Bk|Ex|VDE|Cls0,extract|features|task filter,Nm3|Cal0|Cal|Lambda|Bk|Ex|VDE|VD|VDID0,task filter,Nm3|Cal0|Cal|Lambda|Bk|Ex|Cal0|Nm0,tasks filter,Nm3|Cal0|Cal|Lambda|Bk|Ex|Cal0|Nm2,task filter,Nm3|Cal0|Cal|Lambda|Bk|Ex|Cal0|Nm3,add filter,Nm3|Cal0|Cal|Ex|Bk|Try|Catch|Prm|VDID0,e filter,Nm3|Cal0|Cal|Ex|Bk|Try|Catch|Prm|Cls1,io|exception f,VDID0|Prm|Lambda2|Bk|Ex|VDE|Cls0,extract|features|task f,VDID0|Prm|Lambda2|Bk|Ex|VDE|VD|VDID0,task f,VDID0|Prm|Lambda2|Bk|Ex|VDE|VD|ObjEx|Cls0,extract|features|task f,VDID0|Prm|Lambda2|Bk|Ex|VDE|VD|ObjEx|Nm1,s|command|line|values f,VDID0|Prm|Lambda2|Bk|Ex|VDE|VD|ObjEx|Nm2,f f,VDID0|Prm|Lambda2|Bk|Ex|Cal0|Nm0,tasks f,VDID0|Prm|Lambda2|Bk|Ex|Cal0|Nm2,task f,VDID0|Prm|Lambda2|Bk|Ex|Cal0|Nm3,add f,VDID0|Prm|Lambda2|Cal|Nm3,for|each extract|features|task,Cls0|VDE|VD|VDID0,task extract|features|task,Cls0|VDE|VD|ObjEx|Cls0,extract|features|task extract|features|task,Cls0|VDE|VD|ObjEx|Nm1,s|command|line|values extract|features|task,Cls0|VDE|VD|ObjEx|Nm2,f extract|features|task,Cls0|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks extract|features|task,Cls0|VDE|Ex|Bk|Ex|Cal0|Nm2,task extract|features|task,Cls0|VDE|Ex|Bk|Ex|Cal0|Nm3,add extract|features|task,Cls0|VDE|Ex|Bk|Lambda2|Cal|Nm3,for|each task,VDID0|VD|ObjEx|Cls0,extract|features|task task,VDID0|VD|ObjEx|Nm1,s|command|line|values task,VDID0|VD|ObjEx|Nm2,f task,VDID0|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks task,VDID0|VD|VDE|Ex|Bk|Ex|Cal0|Nm2,task task,VDID0|VD|VDE|Ex|Bk|Ex|Cal0|Nm3,add task,VDID0|VD|VDE|Ex|Bk|Lambda2|Cal|Nm3,for|each extract|features|task,Cls0|ObjEx|Nm1,s|command|line|values extract|features|task,Cls0|ObjEx|Nm2,f extract|features|task,Cls0|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks extract|features|task,Cls0|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm2,task extract|features|task,Cls0|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm3,add extract|features|task,Cls0|ObjEx|VD|VDE|Ex|Bk|Lambda2|Cal|Nm3,for|each s|command|line|values,Nm1|ObjEx|Nm2,f s|command|line|values,Nm1|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks s|command|line|values,Nm1|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm2,task s|command|line|values,Nm1|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm3,add s|command|line|values,Nm1|ObjEx|VD|VDE|Ex|Bk|Lambda2|Cal|Nm3,for|each f,Nm2|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks f,Nm2|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm2,task f,Nm2|ObjEx|VD|VDE|Ex|Bk|Ex|Cal0|Nm3,add f,Nm2|ObjEx|VD|VDE|Ex|Bk|Lambda2|Cal|Nm3,for|each tasks,Nm0|Cal|Nm2,task tasks,Nm0|Cal|Ex|Bk|Lambda2|Cal|Nm3,for|each task,Nm2|Cal|Nm3,add task,Nm2|Cal|Ex|Bk|Lambda2|Cal|Nm3,for|each add,Nm3|Cal|Ex|Bk|Lambda2|Cal|Nm3,for|each for|each,Nm3|Cal|Ex|Bk|Try|Catch|Prm|VDID0,e for|each,Nm3|Cal|Ex|Bk|Try|Catch|Prm|Cls1,io|exception e,VDID0|Prm|Cls1,io|exception e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm0,e e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace e,VDID0|Prm|Catch|Try|Bk|Ex|VDE|VD|VDID0,tasks|results e,VDID0|Prm|Catch|Try|Bk|Ex|VDE|VD|Null1,null e,VDID0|Prm|Catch|Try|Bk|Try|Catch|Prm|VDID0,e e,VDID0|Prm|Catch|Try|Bk|Try|Catch|Prm|Cls1,interrupted|exception io|exception,Cls1|Prm|Catch|Bk|Ex|Cal0|Nm0,e io|exception,Cls1|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace io|exception,Cls1|Prm|Catch|Try|Bk|Ex|VDE|VD|VDID0,tasks|results io|exception,Cls1|Prm|Catch|Try|Bk|Ex|VDE|VD|Null1,null io|exception,Cls1|Prm|Catch|Try|Bk|Try|Catch|Prm|VDID0,e io|exception,Cls1|Prm|Catch|Try|Bk|Try|Catch|Prm|Cls1,interrupted|exception e,Nm0|Cal|Nm2,print|stack|trace void,Cls0|Cls|Cls|VDE|VD|VDID0,tasks|results void,Cls0|Cls|Cls|VDE|VD|Null1,null void,Cls0|Cls|Cls|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks|results void,Cls0|Cls|Cls|VDE|Ex|Bk|Ex|Cal0|Nm3,for|each tasks|results,VDID0|VD|Null1,null tasks|results,VDID0|VD|VDE|Ex|Bk|Try|Catch|Prm|VDID0,e tasks|results,VDID0|VD|VDE|Ex|Bk|Try|Catch|Prm|Cls1,interrupted|exception tasks|results,VDID0|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks|results tasks|results,VDID0|VD|VDE|Ex|Bk|Ex|Cal0|Nm3,for|each null,Null1|VD|VDE|Ex|Bk|Try|Catch|Prm|VDID0,e null,Null1|VD|VDE|Ex|Bk|Try|Catch|Prm|Cls1,interrupted|exception null,Null1|VD|VDE|Ex|Bk|Ex|Cal0|Nm0,tasks|results null,Null1|VD|VDE|Ex|Bk|Ex|Cal0|Nm3,for|each tasks|results,Nm0|As|Cal1|Nm0,executor tasks|results,Nm0|As|Cal1|Nm2,tasks tasks|results,Nm0|As|Cal1|Nm3,invoke|all tasks|results,Nm0|As|Ex|Bk|Try|Catch|Prm|VDID0,e tasks|results,Nm0|As|Ex|Bk|Try|Catch|Prm|Cls1,interrupted|exception tasks|results,Nm0|As|Ex|Bk|Try|Bk|Ex|Cal0|Nm0,executor tasks|results,Nm0|As|Ex|Bk|Try|Bk|Ex|Cal0|Nm2,shutdown tasks|results,Nm0|As|Ex|Bk|Try|Bk|Ex|Cal0|Nm0,tasks|results tasks|results,Nm0|As|Ex|Bk|Try|Bk|Ex|Cal0|Nm3,for|each executor,Nm0|Cal1|Nm2,tasks executor,Nm0|Cal1|As|Ex|Bk|Try|Catch|Prm|VDID0,e executor,Nm0|Cal1|As|Ex|Bk|Try|Catch|Prm|Cls1,interrupted|exception tasks,Nm2|Cal1|Nm3,invoke|all tasks,Nm2|Cal1|As|Ex|Bk|Try|Catch|Prm|VDID0,e tasks,Nm2|Cal1|As|Ex|Bk|Try|Catch|Prm|Cls1,interrupted|exception invoke|all,Nm3|Cal1|As|Ex|Bk|Try|Catch|Prm|VDID0,e invoke|all,Nm3|Cal1|As|Ex|Bk|Try|Catch|Prm|Cls1,interrupted|exception e,VDID0|Prm|Cls1,interrupted|exception e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm0,e e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace e,VDID0|Prm|Catch|Try|Bk|Ex|Cal0|Nm0,executor e,VDID0|Prm|Catch|Try|Bk|Ex|Cal0|Nm2,shutdown e,VDID0|Prm|Catch|Try|Bk|Ex|Cal0|Nm0,tasks|results e,VDID0|Prm|Catch|Try|Bk|Ex|Cal0|Nm3,for|each interrupted|exception,Cls1|Prm|Catch|Bk|Ex|Cal0|Nm0,e interrupted|exception,Cls1|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace interrupted|exception,Cls1|Prm|Catch|Try|Bk|Ex|Cal0|Nm0,executor interrupted|exception,Cls1|Prm|Catch|Try|Bk|Ex|Cal0|Nm2,shutdown interrupted|exception,Cls1|Prm|Catch|Try|Bk|Ex|Cal0|Nm0,tasks|results interrupted|exception,Cls1|Prm|Catch|Try|Bk|Ex|Cal0|Nm3,for|each e,Nm0|Cal|Nm2,print|stack|trace executor,Nm0|Cal|Nm2,shutdown executor,Nm0|Cal|Ex|Bk|Try|Bk|Ex|Cal0|Nm0,tasks|results executor,Nm0|Cal|Ex|Bk|Try|Bk|Ex|Cal0|Nm3,for|each shutdown,Nm2|Cal|Ex|Bk|Try|Bk|Ex|Cal0|Nm0,tasks|results shutdown,Nm2|Cal|Ex|Bk|Try|Bk|Ex|Cal0|Nm3,for|each tasks|results,Nm0|Cal|Lambda|Prm|VDID0,f tasks|results,Nm0|Cal|Lambda|Bk|Try|Bk|Ex|Cal0|Nm0,f tasks|results,Nm0|Cal|Lambda|Bk|Try|Bk|Ex|Cal0|Nm2,get tasks|results,Nm0|Cal|Lambda|Bk|Try|Catch|Prm|VDID0,e tasks|results,Nm0|Cal|Lambda|Bk|Try|Catch|Prm|Unio|Cls0,interrupted|exception tasks|results,Nm0|Cal|Lambda|Bk|Try|Catch|Prm|Unio|Cls1,execution|exception f,VDID0|Prm|Lambda2|Bk|Try|Bk|Ex|Cal0|Nm0,f f,VDID0|Prm|Lambda2|Bk|Try|Bk|Ex|Cal0|Nm2,get f,VDID0|Prm|Lambda2|Bk|Try|Catch|Prm|VDID0,e f,VDID0|Prm|Lambda2|Bk|Try|Catch|Prm|Unio|Cls0,interrupted|exception f,VDID0|Prm|Lambda2|Bk|Try|Catch|Prm|Unio|Cls1,execution|exception f,VDID0|Prm|Lambda2|Cal|Nm3,for|each f,Nm0|Cal|Nm2,get f,Nm0|Cal|Ex|Bk|Try|Catch|Prm|VDID0,e f,Nm0|Cal|Ex|Bk|Try|Catch|Prm|Unio|Cls0,interrupted|exception f,Nm0|Cal|Ex|Bk|Try|Catch|Prm|Unio|Cls1,execution|exception f,Nm0|Cal|Ex|Bk|Try|Bk|Lambda2|Cal|Nm3,for|each get,Nm2|Cal|Ex|Bk|Try|Catch|Prm|VDID0,e get,Nm2|Cal|Ex|Bk|Try|Catch|Prm|Unio|Cls0,interrupted|exception get,Nm2|Cal|Ex|Bk|Try|Catch|Prm|Unio|Cls1,execution|exception get,Nm2|Cal|Ex|Bk|Try|Bk|Lambda2|Cal|Nm3,for|each e,VDID0|Prm|Unio|Cls0,interrupted|exception e,VDID0|Prm|Unio|Cls1,execution|exception e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm0,e e,VDID0|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace e,VDID0|Prm|Catch|Try|Bk|Lambda2|Cal|Nm3,for|each interrupted|exception,Cls0|Unio|Cls1,execution|exception interrupted|exception,Cls0|Unio|Prm|Catch|Bk|Ex|Cal0|Nm0,e interrupted|exception,Cls0|Unio|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace interrupted|exception,Cls0|Unio|Prm|Catch|Try|Bk|Lambda2|Cal|Nm3,for|each execution|exception,Cls1|Unio|Prm|Catch|Bk|Ex|Cal0|Nm0,e execution|exception,Cls1|Unio|Prm|Catch|Bk|Ex|Cal0|Nm2,print|stack|trace execution|exception,Cls1|Unio|Prm|Catch|Try|Bk|Lambda2|Cal|Nm3,for|each e,Nm0|Cal|Nm2,print|stack|trace

[urialon]

Hi @sadiasahar ,
Thank you for your interest in our work!

Can you please specify what did you run, what error did you get, what is your operating system, Java version and python version?

[sadiasahar]

Thank you @urialon for the reply. The operating system I am using is Windows 11 but I am running the code in Google Colab, where
Python version is ---- 3.7.14
Java version is ------- 11.0.16
The hierarchy of my Dataset is
|-- Code2Seq
|--------------Data
|-------------------Train
|-------------------------Class1
|---------------------------------1.java
|---------------------------------2.java

I also changed the directory path in preprocess.sh file and ran the following command

!bash /content/drive/MyDrive/code2seq-master/preprocess_2.sh

the error I got is,

Extracting paths from validation set...
Finished extracting paths from validation set
Extracting paths from test set...
Finished extracting paths from test set
Extracting paths from training set...
dir: /content/drive/MyDrive/code2seq-master/DATA/TRAIN/Class1 was not completed in time
Finished extracting paths from training set
Creating histograms from the training data
subtoken vocab size: 0
node vocab size: 0
target vocab size: 0
File: my_dataset.test.raw.txt
Traceback (most recent call last):
File "/content/drive/MyDrive/code2seq-master/preprocess.py", line 115, in
max_contexts=int(args.max_contexts), max_data_contexts=int(args.max_data_contexts))
File "/content/drive/MyDrive/code2seq-master/preprocess.py", line 53, in process_file
print('Average total contexts: ' + str(float(sum_total) / total))
ZeroDivisionError: float division by zero

[urialon]

can you paste the relevant line from your modified preprocessing script, the one that executes preprocess.py, with the values of it's arguments?

[sadiasahar]

Yes, off course. The line is:

${PYTHON} /content/drive/MyDrive/code2seq-master/preprocess.py --train_data ${TRAIN_DATA_FILE} --test_data ${TEST_DATA_FILE} --val_data ${VAL_DATA_FILE}
--max_contexts ${MAX_CONTEXTS} --max_data_contexts ${MAX_DATA_CONTEXTS} --subtoken_vocab_size ${SUBTOKEN_VOCAB_SIZE}
--target_vocab_size ${TARGET_VOCAB_SIZE} --subtoken_histogram ${SOURCE_SUBTOKEN_HISTOGRAM}
--node_histogram ${NODE_HISTOGRAM_FILE} --target_histogram ${TARGET_HISTOGRAM_FILE} --output_name data/${DATASET_NAME}/${DATASET_NAME}

[sadiasahar]

My appology, I forgot to change the path of JavaExtractor folder assigned to the variable EXTRACTOR_JAR. Now it is Working.

Thanks for the help.

[urialon]

Great! Glad to hear!

[sadiasahar]

thanks again