About Me
Welcome !!!
Hey folks, Welcome to my blog (A tiny technical reference).
I’m an ETL Consultant, hails from Southern Part of India. I’m an enthusiastic and fascinated about latest trends, techniques in Data Warehouse/Business Intelligence area. Data Analyzation, Data Validation and Data Profiling are some of my activities at work.
The main motive of this blog is to provide good technical information. I have reaped some sort of information from different books, websites, blogs etc., and I’m blending the same here in my technical notes. Indeed not only that, I’m going to share my ideas, views, and knowledge that I have gained from my past experiences to educate tyros. I would say this blog is a share point where you’ll find some useful information.
What kind of content you’ll get here?
I emphasized more on basic concepts of Database (DB), Data Warehouse (DW), BO Data Integrator/Data Services (DI/DS) and Best Practices (BP) of related tools in an organized manner.
Notes will be updated regularly and is maintained by Sudheer Sharma, an ETL Consultant. Keep visiting my blog and do not forget to leave your valuable comments.
Hello Sudheer ,
This is some real good work ,trying to help people with your knowledge . It is very informative and meets the cause of maintaining this site. I was strongly recommended by my friend to read this for the basic knowledge of DI.
I really appreciate your effort doing all this . I am business objects developer who is interested in learning DI , It really helps me understand the concepts in a clear way.
Thanks for your efforts and keep up the good work.
Thanks,
murali
Thanks a Million my Brother.
I just started learning BO and seriously searching for DI training. Luckily i got your blog.
And also i am searching for LiveOffice & Xselcius.
If you have any idea, please pass the info my brother.
Thanks and have a great weekend!
regards,
JaY
Hey Sudheer,
i work on BODI and the notes here are awesome… You rock ..
trust me..
Thanks
Your friend
Amandeep
Thanks for your comments friends (Aman, Murali, Jay)
Hi Sudheer,
You are doing great job…keep rocking…
Regards,
S. Thangavel
Thanks for your comments Thangavel
Hi Sudheer
I really appreciate your Knowledge sharing thoughts by creating useful Blogs.I read concepts of
Dwh and Database.I am Beginner notes from this site gave me an clear Idea.
This Informative resource is a real Gift to All.
I wish you All Success frm Raji
Nice content Sudheer, keep posting new things man, you rocks!
Hi Sudheer,
I have a requirement to develop the solution using BODI. The Requirement says as below :
• In case of any errors process owner (vidubs@positiveedge.com) should be notified by email with last 20 lines of error log;
• Location of source files and destination for processed files should be defined as global variables;
• Check for duplicated files and in case of duplicated files it should be moved into “Duplicate” folder and owner should be notified by email;
• In case of corrupted files or wrong format files should be moved into “Unmatched” folder and owner should be notified by email;
• Correctly processed files should be moved in specified network location and organized by subfolder: YEAR\Month (2009\January)
I have completed the below tasks and struck with rest.
• Location of source files and destination for processed files should be defined as global variables;
• Correctly processed files should be moved in specified network location and organized by subfolder: YEAR\Month (2009\January)
I am using the BODI 11.7 version and target as SQL Server 2005.
Could you please provide expertise to solve this issue. If you want me to send the ATL which I worked I am glad to send you.
Thanks in advance.
Regards,
Vidu.
Vidu, here are my reply’s for your questions
• In case of any errors process owner (vidubs@positiveedge.com) should be notified by email with last 20 lines of error log;
–Use mail_to()/smtp_to() function(s) to accomplish this in Catch block, these functions have a feature called adding error lines and trace lines to email notification. “Assuming your mailserver is installed and configured to jobserver”
• Check for duplicated files and in case of duplicated files it should be moved into “Duplicate” folder and owner should be notified by email;
–This can be done using DOS batch scripts/Unix shell scripts, Use commands like FC, cmp etc… once you find any duplicated files then set a Dupe flag/parameter in BODI, If it is 1 then send an email notification else process your load.
• In case of corrupted files or wrong format files should be moved into “Unmatched” folder and owner should be notified by email;
–There are plenty error handling settings on file format editor, options like “capture row format errors”, “write error rows to file” etc.., probably you may need to choose these options and try your luck.
Send your atl file, i will look into it and let you know what needs to be done.
Cheers,
Sudheer
Hi Sudheer,
Thanks for your quick reply. I have developed the the ETL solution based on information provided at your site. It is very informative.
First/Second Point : As you suggested, I will configure mail server and try to send the mail through mail_to()/smtp_to() function(s). Need to explore on the FC, Cmp Dos commands. I will let you know the status.
Third Point : I have set the “capture row format errors”, “write error rows to file” file format options, able to send the duplicate file to overflow file. could not able send each entry of overflow file as mail to supervisor. The control is not moving to catch block, somewhere I am missing the flow. Even I tried you comments on the converting error log file entries into a table.
Could I know your email address, so that i can share more information on my workings. My email address is vidubhushan@yahoo.com.
Thanks,
Vidubhushan.
Great! seems like almost done with your work. Send your findings to godasudheer@dwhnotes.com
PS: dont forgot to mention your atl file version
Excellent explanation!!!
Great site, excellent content….keep up the good work!
Hats off Sudheer
The site is well crafted for beginners like me. Expecting more updates and do keep it live.
Thanks a lot boss…
Sudheer,
Thanks for your effort. would you please share some content of technical document like ( Design document, Mapping and Unit Design document )
Regards,
Manoj.
Thanks for your comments. Yes I could
Cheers,
Sudheer Sharma
Hi Sudheer,
i am from banglore,i want to learn BODI
can u plz suggest m any training institute in banglore.
Example for year : 2010
Day Lite Start
=================
2010.03.14 02:00:00
Day Lite End
=============
2010.11.07 02:00:00
I am converting GMT to CDT/CST [ US Dalls ] Time Zone…based on above values which is given in time zone web sites
case 1 )
myDateTime is 2010.11.07 02:00:00
case 2)
myDateTime is 2010.11.07 02:15:00
How to decide whether myDateTime comes under CDT or CST… ?.
Approach 1 ) inclujding start and end dates
if myDateTime >= Day Lite Start and myDateTime Day Lite Start and myDateTime Day Lite Start and myDateTime <Day Lite End then
TimeZone is CDT
else
TimeZone is CST
I have taken approach 1 … my result/output it for case 1) 2010.11.06 21:00:00 [ CDT ]
case 2 ) 2010.11.06 20:15:00 [ CST]
which is wrong ….
Please tell me which approcah i have to take to decide CDT or CST to get correct timezone
Your requirement may not end with only Texas DST timings I believe. In your case, its better to have a lookup table to find out the DST for the appropriate regions. You start creating a DST lookup table for required regions for the number of years. So that you can do a lookup and decide on what data & time it has to convert. You can set parameters like GMT, locations for the regions.
Hope this helps!
Cheers,
Sudheer Sharma
in above requirement i a trying to convert GMT to CDT/CST..i am unable to decide CDT or CST … Please help me…
Check my above note.
Cheers,
Sudheer Sharma