The Compact Linear Collider (CLIC) was proposed as the next energy-frontier infrastructure at CERN, to study e$^+$e$^-$ collisions at three centre-of-mass energy stages: 380 GeV, 1.5 TeV and 3 TeV. The main goal of its high-energy stages is to search for the new physics beyond the Standard Model (SM). The Inert Doublet Model (IDM) is one of the simplest SM extensions and introduces four new scalar particles: H$^\pm$, A and H; the lightest, H, is stable and hence a natural dark matter (DM) candidate. A set of benchmark points is considered, which are consistent with current theoretical and experimental constraints and promise detectable signals at future colliders. Prospects for observing pair-production of the IDM scalars at CLIC were previously studied using signatures with two leptons in the final state. In the current study, discovery reach for the IDM charged scalar pair-production is considered for the semi-leptonic final state at the two high-energy CLIC stages. Full simulation analysis, based on the current CLIC detector model, is presented for five selected IDM scenarios. Results are then extended to the larger set of benchmarks using the Delphes fast simulation framework. The CLIC detector model for Delphes has been modified to take pile-up contribution from the beam-induced $\gamma\gamma$ interactions into account, which is crucial for the presented analysis. Results of the study indicate that heavy, charged IDM scalars can be discovered at CLIC for most of the proposed benchmark scenarios, with very high statistical significance.