Security analysis of the mode of JH hash function

Recently, NIST has selected 14 second round candidates of SHA3 competition. One of these candidates will win the competition and eventually become the new hash function standard. In TCC'04, Maurer et al introduced the notion of indifferentiability as a generalization of the concept of the indistinguishability of two systems. Indifferentiability is the appropriate notion of modeling a random oracle as well as a strong security criteria for a hash-design. In this paper we analyze the indifferentiability and preimage resistance of JH hash function which is one of the SHA3 second round candidates. JH uses a 2n bit fixed permutation based compression function and applies chopMD domain extension with specific padding. We show under the assumption that the underlying permutations is a 2n-bit random permutation, JH mode of operation with output length 2n - s bits, is indifferentiable from a random oracle with distinguisher's advantage bounded by where σ is the total number of blocks queried by distinguisher. We show that the padding rule used in JH is essential as there is a simple indifferentiablity distinguisher (with constant query complexity) against JH mode of operation without length padding outputting n bit digest. We prove that a little modification (namely chopping different bits) of JH mode of operation enables us to construct a hash function based on random permutation (without any length padding) with similar bound of sponge constructions (with fixed output size) and with same efficiency. On the other hand, we improve the preimage attack of query complexity 2^510.3 due to Mendel and Thompson. Using multicollisions in both forward and reverse direction, we show a preimage attack on JH with n = 512,s = 512 in 2 ⁵⁰⁷ queries to the permutation.