Over the last decades, there has been a rapid growth in the size and complexity of electronic circuits. Since the clock speed of microprocessors is saturated around 3 GHz, computers are no longer able to keep up with the simulation challenges and new simulation approaches are required. SPICE-like simulation algorithms have many intrinsically sequential elements. With the availability of multi-core systems, several attempts have been made to speed up the circuit simulation process, in terms of parallelising the device evaluation or the matrix solution phase. However, these methods have resulted in limited speed-ups or compromised accuracy. Another existing issue is the barrier between device evaluation and matrix solution phases, which prevents the whole simulation process being parallelised. Most of the existing attempts on parallelising circuit simulation algorithm are based on the conventional and coarse grained methods. We propose new very fine-grained parallel approaches for the matrix solution and device evaluation phases with the possibility of mixed analysis of the two phases to totally parallelise the simulation process. Instead of the conventional direct matrix solvers we use highly parallel iterative methods. The motivation behind this approach is the availability of new parallel platforms and architectures for highly parallel and distributed simulations. SpiNNaker project is one of the new architectures which aims to model large-scale spiking neural networks on a massively parallel million-core system. The purpose of this work is proposing very finegrained parallel approaches and preparing the ground work for performing the proposed methods on highly parallel structures. In this work, the matrix solution part of the circuit simulation process is performed using a Jacobi-type iterative method. The proposed fine-grained parallel method distributes the solution of circuit equations across a large number of light-weight processors by allocating one processor to each circuit equation. The device modelling process is inherently a parallel task since modelling each device can be done independently. For the device modelling phase, we use the Secant method instead of conventional Newton-Raphson iterations to avoid the calculation of partial derivatives at each iteration. The proposed methods are applied to a number of benchmark matrices and test circuits and are optimised to work best on sparse systems. Simulation results confirm the functionality of the proposed Jacobi-type iterations in the parallel solution of matrix equations. Compared to a conventional direct matrix solution (LU-factorisation), the proposed fine-grained parallel iterative method performs better as the size of the problem increases with a speed-up of around 2.5x for the largest example matrix. Furthermore, by replacing the computationally intensive Newton-Raphson iterations for the device evaluation phase with the Secant method, which benefits from a much simpler algorithm, the simulation time is improved by a factor of 3 to 4 for the example circuits. Finally, simultaneous evaluation of the proposed parallel iterative method for matrix solution and the Secant method for device evaluation, to replace the conventional direct solution and Newton-Raphson iterations, resulted in a significant improvement in the simulation time of the under test circuits. The simulation results suggest that compared to the conventional sequential algorithm, the proposed fine-grained parallel approach has achieved an overall speed-up of 16x and 22x for the two test circuits.