I just started with this project, is there any way of monitoring the battery capacity of a battery with just the use of rtc, voltage sensor and current sensor?
For lead acid batteries, voltage levels measured when the battery is not under load (open circuit) are often acceptable indicators of charge state. See BU-903: How to Measure State-of-charge - Battery University
That is when the battery is not under the load. But the monitoring I want to do is continuous(charging/discharging state) and then the display will be in LCD. I read about the state of charge of battery but this state is based on the voltage of the battery, >12.7* is considered as 100% and <10.5 is considered as dead. Is it possible to measure the voltage of the battery while its on charging/discharging state?(what I remember is not) Or is there any other* way of knowing the state of charge of battery?
Thank you for the reply 
sorry kinda new here.
In theory you have the makings, how practical it is I don't know.
Measure the V and I (with direction) so you have P at any time, and you know from I's direction if that's charging or discharging. Integrate that P over time and you have energy in or out. If you know the energy in the battery to start with, you will always know the energy in the battery and hence its state of charge.
The problem is it will be always in charging state, and it will only be off* the discharging state if and only if all loads are removed. What I am trying to monitor is a battery that is connected in both the charger(solar panel) and its load.
Do you want to measure Capacity or State of Charge (SoC).
If you are measuring Capacity what are you wanting to achieve?
If you just want to know if "X" battery will run "Y" load for "Z" time period you must test it by running Y load until either you reach Z Time or the battery discharges below minimum voltage.
If you want to know if the battery is getting close to end of life then the most common test is to discharge at a fixed known current, to a set minimum Volts per Cell (VPC) level, then comparing this with the manufacturers specification for the battery. This is the testing that most Utility services in N.Z. do to maintain their batteries.
State of Charge can tell you how charged the battery is but not how long the battery will support a given load.
Minimum equipment for Capacity testing would be RTC, and Voltage Sensing with the ability to stop the test if the battery drops below minimum voltage, you will also need a known load. The cut-off voltage could be the lowest voltage your equipment will work at, but most often will be the minimum voltage the battery can deliver without significantly damaging the battery or reducing its life expectancy.and a known load,
What I am trying to monitor is a battery that is connected in both the charger(solar panel) and its load.
Reply #3 is the correct answer. Monitor the current into and out of the battery, paying attention to the sign.
Dehnyz:
The problem is it will be always in charging state, and it will only be on discharging state if and only if all loads are removed. What I am trying to monitor is a battery that is connected in both the charger(solar panel) and its load.
Sorry but that seems incorrect, the battery will discharge whenever there is no Solar pressure charging the battery, all Lead Acid Batteries self discharge with no load connected, and of course if the load is present and the Solar isn't then of course the battery will be discharging.
You could try Googling "Coulomb Counting" which may help.
Dehnyz:
I just started with this project, is there any way of monitoring the battery capacity of a battery with just the use of rtc, voltage sensor and current sensor?
I used to design UPSs for a living, and yes: you can estimate the capacity of a battery with just time, current, and voltage.
The battery university link posted above is a good one to start. Read the Coulomb Counting and Voltage Method sections. The vast majority of SoC algorithms in the world are a combination of the two.
Coulomb Counting will be the basis of your SoC algorithm. You know the capacity (Ah rating) of your battery. If you can accurately measure current and time, you can integrate the current over time to determine the SoC. For example, if you start at 100% SoC on a 7Ah battery, and you measure a 1A discharge for 30 minutes, your SoC is now (7Ah - 1A * 0.5h)/7Ah = 92.9% SoC. If you start at 0% SoC on the same battery and measure a 0.5A charge for 5 hours, your SoC is (0.5A * 5h)/7Ah = 35.7%.
If your current measurement and your capacity rating were perfect you could do nothing but coulomb count. Your current measurement is going to have error, however, making your coulomb count drift from the actual over time. Your capacity rating isn't perfect either: batteries usually have more than their Ah rating when they're new and that capacity fades over time. Complicating things more, batteries can not supply the same capacity at all loads. A battery discharged at full load will last less than half as long than the same battery discharged at half load. That complicates the otherwise simple equation of SoC = (couloumb_count/capacity), since capacity is a function of load.
To deal with this, SoC algorithms will use a Voltage Method to correct the Coulomb Count. Most battery datasheets will offer discharge curves that show the voltage over discharge at several loads. Figure 1 in the battery university link shows examples of these where: 1) the voltage is very flat in the middle, 2) the voltage changes rapidly at the top and bottom, and 3) there is a complex impedance relating how much the battery voltage droops under different loads. Figure 1 is for LiFePO4 which has one of the flattest battery discharge curves, but even Lead Acid has a similar profile.
There are different ways to put the battery discharge information in code to estimate the SoC, but a common one is to store the Voltage vs SoC under light/no load and a model of the internal impedance of the battery. When your product is charging and discharging, you first adjust the voltage reading by your measured current * your modeled battery impedance to estimate an open circuit voltage. You can then look up that open circuit voltage in the Voltage vs SoC under light/no load to estimate the SoC. This voltage based SoC is going to be very inaccurate in the flat part of the discharge curve. It is also going to be inaccurate when current is flowing since a battery's impedance is much more complicated than you'll be able to model in FW. The Voltage Method will be useful at the beginning and end of discharge, however, to determine when your Coulomb Count has drifted from the actual value. You then use the Voltage Method SoC to adjust your Coulomb Count such that it ends at 0% at the end of discharge and 100% at the end of charge.
A note about counting kWh instead of Ah: don't count kWh...count Ah. Conservation of energy and conservation of charge are both laws of physics. For you to estimate SoC with kWh, however, you need to know the energy efficiency for the battery (which is pretty much impossible). The coulombic efficiency of the battery is going to be very close to 100%, however. Think of a battery as being modeled by a voltage source in series with a small impedance and in parallel with a large impedance. As you charge/discharge the battery you are going to lose some energy in the series impedance but there is no simple way to know how much. The parallel impedance, however, is going to be many MegaOhms (it's what models self-discharge, which is typically measured in %/month). As current goes into your battery a little will go through this parallel impedance (you can sort of think of it like it's still self discharging even though you're charging it), but for all useful purposes you can assume that it's all going into the cell. This is why you see SoC algorithms using "Coulomb Counting" instead of "Joule Counting."
Hope this helps and hope it wasn't too much info.
Thank you for all of your replies. It's been very helpful. I will be studying all of your answer. I'll be hoping for your responses if I have an question again. 
Thanks.
I hope you will have got the impression from @BigBobby's extensive reply that this is not a simple process.
And if you are using the battery (as opposed to just having it as a fully-charged standby for occasional use, like a UPS) I reckon you will be lucky to be within 20% of the right answer after a week or so. As the battery ages your ability to keep track will deteriorate further.
My advice would be to ignore any attempt at fine measurement and just set high and low voltage levels which are used to trigger an alarm or appropriate defensive actions.
All this based on relying on a 200Ah lead acid battery for my every-day source of electricity - charged by a diesel engine and solar panels (to reduce the diesel consumption).
And assume that the new capacity of your battery is just 50% of the sticker value.
...R
Robin2:
I hope you will have got the impression from @BigBobby's extensive reply that this is not a simple process.And if you are using the battery (as opposed to just having it as a fully-charged standby for occasional use, like a UPS) I reckon you will be lucky to be within 20% of the right answer after a week or so. As the battery ages your ability to keep track will deteriorate further.
Searching google lately, I get a million pages about using Kalman filters to improve state-of-charge (and I think capacity too?) estimation.
I admit that I have no experience with Kalman filters. It seems from the forums, however, that it's not out of the realms of the Ardunio/hobbyist?
Most of what you read on hobby forums about Kalman filters is wrong.
In order to work properly, each filter needs to be designed for a specific system model, which also includes an accurate model for the various noise sources. That means that there is no general purpose code available, and much of the code that is "floating around" is simply wrong.
It seems possible that for something like a battery system, a reasonably general Kalman filter model could be formulated, but I haven't looked into it.
For reading I recommend "Optimal State Estimation" by Dan Simon. It is an excellent and very readable textbook, intended for engineering students. Simon gives an example for general readership in this overview, but the notation is not standard and some rather indefensible choices are made.
BigBobby:
Searching google lately, I get a million pages about using Kalman filters to improve state-of-charge (and I think capacity too?) estimation.
The big problem with sophisticated (and simple) systems for calculating the state of charge of a battery is "how do you prove that the results are correct?"
If you discharge the battery by 100% just to see how much energy was in it you will almost certainly shorten its life - and it takes a very long time, when you allow for a subsequent full charge taking 8 or 10 hours. And you would need to do it several times to get an average. Which would probably ruin the battery.
...R
There is an interesting project here
I've built this slightly modified it to use an AD1115 and a shunt with op-amp and been bench testing it and the results seem fairly accurate. The way I tested it was simulating it the AH set to 105AH and constant load (Set with pot's) and set an alarm once it has only 30% left the timing seemed correct compared to the calculated result, I know this could be completely different in real life depending on the age of the battery or how good it is. Not tried the charging part has this would be harder because of the charge rate and voltage raising. Not done much more but from the reading on the forum sounds very positive
might be worth a look