If you want to do it by the UI, first choose the Visualization as Line Chart and then select Null Values as Connect. Then change to any other chart that you want to use and use Chart Overlay.
Alternatively you can add the following to Dashboard in Simple XML:
<option name="charting.chart.nullValueMode">connect</option>
Updated answers I have attached screenshot of a Chart with Overlaid fields AvgOverall...
<option name="charting.chart.overlayFields">AvgOverall</option>
As you can see Based on Null Value Mode - Gaps, Connect and Zero, I can get different interpretation of series where ever data is not present or null.
This does not work for overlay chart. It only works for regular line chart.
@vaibhavagg2006 I have updated my answers and added a screenshot to show that nullValueMode is applicable to Overlaid fields also.
Please let me know if I have misunderstood your question in anyway. In case I have please add current Screenshot and if possible the expected behavior.
Just add this to that chart section in the XML source will do:
connect