Almost all (of course not) you need to knowabout Concrete poetry you can find in this documentary at ubuweb.
The film documents the Sackner Archives of Visual and Concrete Poetry, and gives at least an idea of the special relationship that can be established between reader and poet.
As I see it, the problem is not so much „is concrete poetry literature” as „is literature only literature?”.
As for where poetry starts and where sculpture for an example replaces it, here it is the work of Michalis Pichler (2008/2009).
The description goes
„UN COUP DE DÉS JAMAIS N‘ABOLIRA LE HASARD (sculpture)
Un Coup de Dés Jamais N’Abolira Le Hasard (A Throw of the Dice will Never Abolish Chance) is a book by Michalis Pichler published 2008/09 in Berlin. The work is a close copy of the 1914 edition of the french symbolist poet Stéphane Mallarmé’s poem of the same name, but with all the words cut out by laser, in a way that corresponds directly to the typographic layout used by Mallarmé to articulate the text*.
A Preface features the entire poem written as a block of text with each line separated by a slash (/). This block-transcription of the Mallarme-Text was carried out in 1969 by Broodthaers.
12 double spreads follow, with immaculately cut out windows standing in for the text. When turning the pages, numerous shadows are being generated by the cutouts.
The work is soft bound and feels quite insubstantial. The cover is a facsimile of the original cover as published in 1914 by Librairie Gallimard, with the word ‘SCULPTURE’ replacing ‘POEME’ (or ‘IMAGE’) in the centre of the design.
500 copies are produced on paper, 90 on translucent paper and 10 on acrylic glass plates.
* Mallarmé had written the poem in 1897, also saw it published in a magazine called Cosmopolis, but left copious notes as to how it should be typeset, instructions that were finally carried out 16 years after his death, in 1914.
In 1969 this work was appropriated by notorious Marcel Broodthaers, who replaced the words by black stripes. He replaced the classification ‘POEME’ on the cover by the word ‘IMAGE’.”
In fact the whole poem-image-sculpture follows the changing form of expression constituted only in relation with the context that is invoked for each act.